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METHOD AND APPARATUS FOR PROVIDING 
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5 

Description 

Technical Field 

The present invention relates generally to interface devices between humans and 
computers, and more particularly to computer input/output devices that provide force feedback to 
10 the user. 

Background Art 

Virtual reality computer systems provide users with the illusion that they are part of a 
"virtual" environment. A virtual reality system will typically include a computer processor, such 
as a personal computer or workstation, specialized virtual reality software, and virtual reality I/O „ 
15 devices such as head mounted displays, sensor gloves, three dimensional ("3D") pointers, etc. 

One common use for virtual reality computer systems is for training. In many fields, 
such as aviation and vehicle and systems operation, virtual reality systems have been used 
successfully to allow a user to learn from and experience a realistic "virtual" environment. The 
appeal of using virtual reality computer systems for training relates, in part, to the ability of such 

20 systems to allow trainees the luxury of confidently operating in a highly realistic environment and 
making mistakes without "real world" consequences. Thus, for example, a trainee pilot or 
automobile driver can learn to operate a vehicle using a virtual reality simulator without concern 
for accidents that would cause injury, death and/or property damage in the real world. Similarly, 
operators of complex systems, e.g., nuclear power plants and weapons systems, can safely 

25 practice a wide variety of training scenarios that would risk life or property i r performed in 
reality. 

For example, a virtual reality computer system can allow a doctor-trainee or other human 
operator or user to "manipulate" a scalpel or probe within a computer-simulated "body", and 
thereby perform medical procedures on a virtual patient. In this instance, the I/O device which is 
30 typically a 3D pointer, stylus, pr the like is used to represent a surgical instrument such as a 
scalpel or probe. As the "scalpel" or "probe" moves within a provided space or structure, results 
of such movement are updated and displayed in a body image displayed on the screen of the 
computer system so that the operator can gain the experience of performing such a procedure 
without practicing on an actual human being or a cadaver. 
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In other applications, virtual reality computer systems allow a user to handle and 
manipulate the controls of complicated and expensive vehicles and machinery. For example, a 
pilot or astronaut in training can operate a fighter aircraft or spacecraft by manipulating controls 
such as a control joystick and other buttons and view the results of controlling the aircraft on a 
5 virtual reality simulation of the aircraft in flight. In yet other applications, a user can manipulate 
objects and tools in the real world, such as a stylus, and view the results of the manipulation in a 
virtual reality world with a "virtual stylus" viewed on a screen, in 3-D goggles, etc. 

For virtual reality systems to provide a realistic (and therefore effective) experience for 
the user, sensory feedback and manual interaction should be as natural as possible. As virtual 

10 reality systems become more powerful and as the number of potential applications increases, 
there is a growing need for specific human/computer interface devices which allow users to 
interface with computer simulations with tools that realistically emulate the activities being 
represented within the virtual simulation. Such procedures as laparoscopic surgery, catheter 
insertion, and epidural analgesia should be realistically simulated with suitable human/computer 

1 5 interface devices if the doctor is to be properly trained. Similarly, a user should be provided with 
a realistic interface for manipulating controls or objects in a virtual reality simulation to rt gain 
useful experience. 

While the state of the an in virtual simulation and medical imaging provides a rich and 
realistic visual feedback, there is a great need for new human/computer interface tools which 
20 allow users to perform natural manual interactions with the computer simulation. For medical 
simulation, there is a strong need to provide doctors with a realistic mechanism for performing 
the manual activities associated with medical procedures while allowing a computer to accurately 
keep track of their actions. There is also a need in other simulations to provide virtual reality 
users with accurate and natural interfaces for their particular tasks. 

25 In addition to sensing and tracking a user's manual activity and feeding such information 

to the controlling computer to provide a 3D visual representation to the user, a human interface 
mechanism should also provide force or tactile ("haptic") feedback to the user. The need for the 
user to obtain realistic tactile information and experience tactile sensation is extensive in many 
kinds of simulation. For example, in medical/surgical simulations, the "feel" of a probe or 

30 scalpel simulator is important as the probe is moved within the simulated body. It would 
invaluable to a medical trainee to learn how an instrument moves within a body, how much force 
is required depending on the operation performed, the space available in a body to manipulate an 
instrument, etc. In simulations of vehicles or equipment, force feedback for controls such as a 
joystick can be necessary to realistically teach a user the force required to move the joystick when 

35 steering in specific situations, such as in a high acceleration environment of an aircraft. In virtual 
world simulations where the user can manipulate objects, force feedback is necessary tc 
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realistically simulate physical objects; for example, if a user touches a pen to a table, the user 
should feel the impact of the pen on the table. An effective human interface not only acts as an 
input device for tracking motion, but also as an output device for producing realistic tactile 
sensations. A "high bandwidth" interface system, which is an interface that accurately responds 
5 to signals having fast changes and a broad range of frequencies as well as providing such signals 
accurately to a control system, is therefore desirable in these and other applications. 

There are number of devices that are commercially available for interfacing a human with 
a computer for virtual reality simulations. There are, for example, 2-dimensional input devices 
such as mice, trackballs, and digitizing tablets. However, 2-dimensional input devices tend to be 
10 awkward and inadequate to the task of interfacing with 3-dimensional virtual reality simulations. 

Other 3-dimensional interface devices are available. A 3-dimensional human/computer 
interface tool sold under the trademark Immersion PROBE 1 " 1 is marketed by Immersion Human 
Interface Corporation of Santa Clara, California, and allows manual control in 3-dimensional 
virtual reality computer environments. A pen-like stylus allows for dexterous 3-dimensional 
15 manipulation, and the position and orientation of the stylus is communicated to a host computer. 
The Immersion PROBE has six degrees of freedom which convey spatial coordinates (x, y, z) * 
and orientation (roll, pitch, yaw) of the stylus to the host computer. 

While the Immersion PROBE is an excellent 3-dimensional interface tool, it may be 
inappropriate for certain virtual reality simulation applications. For example, in some of the 

20 aforementioned medical simulations three or four degrees of freedom of a 3-dimensional 
human/computer interface tool is sufficient and, often, more desirable than five or six degrees of 
freedom because it more accurately mimics the real-life constraints of the actual medical 
procedure. More importantly, the Immersion PROBE does not provide force feedback to a user 
and thus does not allow a user to experience an entire sensory dimension in virtual reality 

25 simulations. 

In typica.' multi-degree of freedom apparatuses that include force feedback, there are 
several disadvantages. Since actuators which supply force feedback tend to be heavier and larger 
than sensors, they would provide inertial constraints if added to a device such as the Immersion 
PROBE. There is also the problem of coupled actuators. In a typical force feedback device, a 

30 serial chain of links and actuators is implemented to achieve multiple degrees of freedom in a 
desired object positioned at the end of the chain, i.e., each actuator is coupled to the previous 
actuator. The user who manipulates the object must carry the inertia of all of the subsequent 
actuators and links except for the first actuator in the chain, which is grounded. While it is 
possible to ground all of the actuators in a serial chain by using a complex transmission of cables 

35 or belts, the end result is a low stiffness, high friction, high damping transmission which 
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corrupts the bandwidth of the system, providing the user with an unresponsive and inaccurate 
interface. These types of interfaces also introduce tactile "noise" to the user through friction and 
compliance in signal transmission and limit the degree of sensitivity conveyed to the user through 
the actuators of the device. 

5 Other existing devices provide force feedback to a user. In U.S. Patent 5,184,3 19, by J. 

Kramer, an interface is described which provides force and texture information to a user of a 
computer system. The interface consists of an glove or "exoskeleton" which is worn over the 
user's appendages, such as fingers, arms, or body. Forces can be applied to the user s 
appendages using tendon assemblies and actuators controlled by a computer system to simulate 

10 force and textual feedback. However, the system described by Kramer is not easily applicable to 
simulation environments such as those mentioned above where an object is referenced in 3D 
space and force feedback is applied to the object. The forces applied to the user in Kramer arc 
with reference to the body of the user; the absolute location of the user's appendages are not 
easily calculated. In addition, the exoskeleton devices of Kramer can be cumbersome or even 

15 dangerous to the user if extensive devices are worn over the user's appendages. Furthermore, 
the devices disclosed in Kramer are complex mechanisms in which many actuators must be used 
to provide force feedback to the user. 

In other situations, low-cost and portable mechanical interfaces having force feedback are 
desirable. Active actuators, such as motors, generate forces on an interface device and the user 

20 manipulating the interface device so that the interface device can move independently of the user. 
While active actuators often provide quite realistic force feedback, they can also be quite bulky 
and typically require large power supplies to operate. In addition, active actuators typically 
require high speed control signals to operate effectively and provide stability. In many 
situations, such high speed control signals and high power drive signals arc not available or too 

25 costly, especially in the competitive, low-cost market of personal computers. Furthermore, 
active actuators can sometimes prove unsafe for a user when strong, unexpected forces are 
generated on a user of the interface who does not expect those forces. 

Therefore, a less complex, more compact, and less expensive alternative to a 
human/computer interface tool having force feedback, lower inertia, higher bandwidth, and less 
30 noise is . desirable for certain applications. A less expensive interface requiring slower 
communication signals and being safer for the user is also needed. 
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Disclosure of the Invention 

The present invention provides a human/computer interface apparatus which can provide 
from two to six degrees of freedom and highly realistic force feedback to a user of the appararus. 
The preferred apparatus includes a gimbal mechanism and linear axis member which provide 
5 three degrees of freedom to an object coupled to the apparatus and held by the user. The 
structure of the apparatus permits transducers to be positioned such that their inertial contribution 
to the system is very low. In addition, a capstan drive mechanism provides mechanical 
advantage in applying force feedback to the user, smooth motion, and reduction of friction, 
compliance, and backlash of the system. The present invention is particularly well suited to 

10 simulations of medical procedures using specialized tools and moving an object such as a stylus 
or joystick in three-dimensional simulations. Another embodiment of the preferred apparatus 
includes a transducer system including passive actuators that require lower power and slower 
control signals than active actuators. A desired amount of play, such as compliance or backlash, 
is preferably introduced in the system to allow a computer system to effectively control an object 

15 connected to the interface and transducer system. 

An apparatus of the present invention for interfacing the motion of an object with an " 
electrical system includes a gimbal mechanism that provides two rcvolute degrees of freedom to 
an object about two axes of rotation. In the preferred embodiment, the gimbal mechanism is a 
closed loop five-member linkage including a ground member coupled to a ground surface, first 
20 and second extension members, each being coupled to the ground member, and first and second 
central members, the first central member having an end coupled to the first extension member 
and the second central member having an end coupled to the second extension member. 

A linear axis member is coupled to the gimbal mechanism at the intersection of the two 
central members, which is at the intersection of the two axes of rotation. The linear axis member 
25 is capable of being translated along a third axis to provide a third degree of freedom. The user 
object is coupled to the linear axis member and is thus translatable along the third axis so that the 
object can be moved along all three degrees of freedom. Transducers are also coupled between 
members of the gimbal mechanism and linear axis member to provide an electromechanical 
interface between the object and the electrical system. 

30 In one embodiment, the linear axis member can be rotated about its lengthwise axis to 

provide a fourth degree of freedom. Four transducers are preferably provided, each transducer 
being associated with a degree of freedom. The transducers for the first three degrees of freedom 
include sensors and actuators, and the transducer for the fourth degree of freedom preferably 
includes a sensor. The sensors are preferably digital encoders and the actuators are basket 

35 wound DC servo motors. The sensors sense the positions of the object along the respective 



WO 96/22591 



PCTAJS96/00701 



degrees of freedom and provide the sensory information to a digital processing system such as a 
computer. The actuators impart forces along the respective degrees of freedom in response to 
electrical signals produced by the computer. 

In the preferred embodiment, a capstan drive mechanism is coupled between an actuator 
5 and the gimbal mechanism for each degree of freedom of the gimbal mechanism. The capstan 
drive mechanism transmits the force generated by the transducer to the gimbal mechanism and 
transmits any forces generated by the user on the gimbal mechanism to the transducer. In 
addition, a capstan drive mechanism is preferably used between the linear axis member and a 
transducer to transmit force along the third degree of freedom. The capstan drive mechanisms 
10 each preferably include a rotating capstan drum rotatably coupled to the gimbal mechanism, 
where the capstan drum is coupled to a pulley by a cable and the transducer is coupled to the 
pulley. 

In another embodiment, a floating gimbal mechanism is coupled to the linear axis member 
to provide fifth and sixth degrees of freedom to an object coupled to the floating gimbal 
15 mechanism. Fifth and sixth degree of freedom transducers are coupled to the floating gimbal 
mechanism to sense the position of the object along the fifth and sixth degrees of freedom: In 
one embodiment, the handle or grip of a medical tool such as a laparoscope is used as the object 
in a medical procedure simulation. In other embodiments, a stylus or a joystick is used as the 
object. 

20 Another apparatus embodiment of the present invention uses passive actuators. The 

apparatus includes a sensor that is preferably coupled to the object an which detects movement of 
an object along a degree of freedom. The sensor has a sensing resolution, and preferably an 
amount of play less than the sensing resolution exists between the sensor and the object. More 
preferably, an amount of play that is an order of magnitide less than the sensing resolution, or a 

25 negligible amount of play, exists between the sensor and object. 

The apparatus also include- an actuator assembly that includes an actuator coupled to the 
object to transmit a force to the object along the degree of freedom. The actuator is preferably an 
electromechanical passive resistance element, such as a magnetic particle brake. The actuator 
assembly also includes a play mechanism that is coupled to the actuator for providing a desired 
30 amount of play between the actuator and the object along the degree of freedom. The desired 
amount of play is greater than the sensing resolution of the sensor so that the sensor can detect 
the play. Such desired play can include torsion flex (compliance) or rotary backlash. When the 
play is provided as rotary backlash, the actuator is preferably coupled to a coupling having a 
keyed bore which is smaller than a keyed shaft that is received by the keyed bore. The actuator 
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and the sensor provide an electromechanical interface between the object and the electrical 
system. 

Other embodiments of the apparatus include a gimbal mechanism that provides a first 
rcvolute degree of freedom to an object engaged with the gimbal mechanism about an axis of 
5 rotation. The gimbal mechanism preferably includes a closed loop five member linkage. A 
sensor, having a sensing resolution, is rigidly coupled to the gimbal mechanism for sensing 
positions of the object along the first degree of freedom. A braking mechanism is coupled to the 
gimbal mechanism to create a drag along the first degree of freedom and provides a desired 
amount of play between the actuator and the object along the degree of freedom. The desired 
10 amount of play is equal to or greater than the sensor's sensing resolution. The actuator and the 
sensor provide an electromechanical interface between the object and an electrical system. 

The apparatus further includes a sensor and braking mechanism for sensing and 
providing force feedback along a second degree of freedom provided by the gimbal mechanism. 
The braking mechanism includes an acutator and coupling to provide the desired amount of play. 

15 A capstan drive mechanism is coupled between the actuator and the gimbal mechanism. The 
capstan drive mechanism transmits the force generated by the actuator to the gimbal mechanism " 
and transmits forces applied to the gimbal mechanism by a user to the sensor. A linear axis 
member can also be coupled to the gimbal mechanism at the intersection of the two axes of 
rotation. The object is coupled to the linear axis member and the linear axis member and object 

20 can be translated along a third axis in a third degree of freedom. Transducers can also be 
included for sensing positions of said object and/or creating a drag along third, fourth, fifth, and 
sixth degrees of freedom. The object can be a surgical tool, a stylus, a joystick, or similar 
articles. A method for interfacing motion of an object with an electrical system includes steps 
similar to the elements described above. 

25 In yet another embodiment of the present invention, a system for controlling an 

electromechanical interface apparatus manipulated by a user includes a digital computer system 
for receiving an input control signal and for providing an output control signal which updates a 
process in response to the input control signal. A passive actutator for receiving the output 
control signal provides a resistive force along a degree of freedom to an object coupled to the 

30 passive actuator. The object is preferably grasped and moved by the user. The resistive force is 
based on information in the output control signal and resists a force applied to the object by the 
user along the degree of freedom. A sensor detects motion of the object and outputs the input 
control signal including information representative of the position and motion of the object to the 
digital computer system. Preferably, the digital computer updates a simulation process in 

35 response to the input control signal and displays a simulation to the user on a display screen. A 
play mechanism preferably provides a desired amount of play between the actuator and the 
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object, the desired amount of play being greater than a sensing resolution of the sensor. A serial 
interface can output the output control signal from the computer system and can receive the input 
control signal to the computer system. A digital to analog converter can receive the output 
control signal, convert the output control signal to an analog control signal, and output the analog 
5 control signal to the passive actuator. Finally, a microprocessor can provide the ouput control 
signal from the serial interface to the digital to analog converter and can receive the input control 
signal from the sensor. 

In another method for controlling an interface apparatus according to the present 
invention, steps include sensing the current position of an object coupled to an interface 

10 apparatus and determining the difference between the current position of the object and a 
previous position of the object. A magnitude of a resistive force to be applied to the object is 
determined; this magnitude is based at least in part on the difference between the current postion 
and the previous position. A control signal is provided to a passive actuator to transmit a 
resistive force having the determined magnitude to the object. The above steps are repeated as 

15 the user moves the object. The current position of the object is preferably sensed even when the 
object is locked into a position by the passive actuator. Preferably, a damping constant is 
determined which is multiplied by the difference to determine the magnitude of the resistive 
force. 

The gimbal mechanism of the present invention provides a structure allowing transducers 
20 associated with two degrees of freedom to be decoupled from each other and instead be coupled 
to a ground surface. This allows the weight of the transducers to contribute a negligible inertia to 
the system, providing a low friction, high bandwidth motion system. The addition of a linear 
axis member and transducer positioned near the center of rotation of the gimbal mechanism 
allows a third degree of freedom to be added with minimal inertia. The present invention also 
25 includes capstan drive mechanisms coupled between the transducers and moving components of 
the apparatus. The capstan drive provides mechanical advantage while allowing smooth 
movement to be achieved and providing negligible friction and backlash to the system. These 
advantages allow a computer system to have more complete and realistic control over force 
feedback sensations experienced by a user of the apparatus. 

30 The passive actuator embodiment of the present invention requires less power and slower 

control signals than active actuator embodiments. A desired amount of play, such as backlash or 
compliance, is provided between an actuator and an interfaced user object so that a controlling 
computer can determine the direction that a user moves the object, even when the passive 
actuators are holding the object stationary. In addition, the user preferably cannot feel the play in 

35 the system. The transducer system can be used on a variety of mechanical interfaces providing 
one to six degrees of freedom and can also be used with capstan drive mechanisms so that the 
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desired play is substantially the only play introduced to the interface system. These 
improvements allow a computer system to have more complete and accurate control over a low- 
cost passive interface providing realistic force feedback. 

These and other advantages of the present invention will become apparent to those skilled 
in the art upon a reading of the following specification of the invention and a study of the several 
figures of the drawing. 
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Brief Description of the Drawings 

Figure 1 is a perspective view of a virtual reality system which employs an apparatus of 
the present invention to interface a laparoscope tool handle with a computer system; 

Figure 2 is a schematic diagram of a mechanical apparatus of the present invention for 
5 providing mechanical input and output to a computer system; 

Figure 3 is a perspective front view of a preferred embodiment of the mechanical 
apparatus of Figure 2; 

Figure 4 is a perspective rear view of the embodiment of the mechanical apparatus of 
Figure 3; 

10 Figure 5 is a perspective detailed view of a capstan drive mechanism used for two 

degrees of motion in the present invention; 

Figure 5a is a side elevational view of the capstan drive mechanism shown in Figure 5\ 

Figure 5b is a detailed side view of a pulley and cable of the capstan drive mechanism of 
Figure 5; 

15 Figure 6 is a perspective view of a center capstan drive mechanism for a linear axis 

member of the mechanical apparatus shown in Figure 3; 

Figure 6a is a cross sectional top view of a pulley and linear axis member used in the 
capstan drive mechanism of Figure 6; 

Figure 6b is a cross sectional side view of the linear axis member and transducer shown 
20 in Figure 6; 

Figure 7 is a perspective view of an embodiment of the apparatus of Figure 2 having a 
stylus object for the user; 

Figure 8 is a perspective view of an embodiment of the apparatus of Figure 2 having a 
joystick object for the user, 

25 Figure 9 is a block diagram of a computer and the interface between the computer and the 

mechanical apparatus of Figure 2; 

Figure 10 is a schematic diagram of a suitable circuit for a digital to analog controller of 
the interface of Figure 9; 
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Figure 1 1 is a schematic diagram of a suitable power amplification circuit for powering 
the actuators of the present invention as shown in Figure 9; 

Figure 12a is a schematic diagram of a transducer system in accordance with the present 
invention; 

Figure 12b is a schematic diagram of an alternate embodiment of the transducer system of 
Figure 12a; 

Figure 13 is a schematic diagram of the transducer system of Figure 12a which provides 
backlash between an actuator and an object; 

Figure 14a is a sectional side view of the actuator shaft and coupling of the transducer 
system of Figure 13; 

Figure 14b is a sectional side view of the actuator shaft and coupling of Figure 14a; 

Figure 15 is a detailed view of the keyed portions of the actuator shaft and coupling of 
Figure 14a; 

Figure 16 is a schematic diagram of the transducer system of Figure 12a having a flexible 
coupling; 

Figure 17 is a schematic diagram of the transducer systems of Figures 12a and 12b 
coupled to the mechanical apparatus of Figure 2; 

Figure 18 is a perspective view of the transducer systems of Figures 12a and 12b coupled 
to the mechanical apparatus of Figure 8; 

Figure 19 is a perspective view of a slotted yoke mechanical apparatus used with the 
transducer system of Figure 12a; 

Figure 20 is a block diagram showing an interface for a mechanical apparatus having the 
transducer system of Figure 12a; 

Figure-21 is a flow diagram illustrating a method for controlling an actuator of the 
transducer system of Figure 12a in the simulation of a fluid environment; and 

Figure 22 is a flow diagram illustrating a method for controlling an actuator of the 
transducer system of Figure 12a when encountering an obstacle in a virtual environment. 
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Rest Modes fo r Carrvinp out the Invention 

In Figure 1, a virtual reality system 10 used to simulate a medical procedure includes a 
human/computer interface apparatus 12, an electronic interface 14, and a computer 16. The 
illustrated virtual reality system 10 is directed to a virtual reality simulation of a laparoscopic 
5 surgery procedure. The software of the simulation is not a part of this invention and thus will 
not be discussed in any detail. However, such software is commercially available as, for 
example, Teleos™ from High Techsplanations of Rockville, Maryland. Suitable software 
drivers which interface such simulation software with computer input/output (I/O) devices are 
available from Immersion Human Interface Corporation of Santa Clara* California. 

10 The handle 26 of a laparoscopic tool 18 used in conjunction with the present invention 

is manipulated by an operator and virtual reality images are displayed on a screen 20 of the digital 
processing system in response to such manipulations. Preferably, the digital processing system 
is a personal computer or workstation, such as an IBM-PC AT or Macintosh personal computer, 
or a SUN or Silicon Graphics workstation. Most commonly, the digital processing system is a 

15 personal computer which operates under the MS-DOS operating system in conformance with an 
IBM PC AT standard. 

The human/interface apparatus 12 as illustrated herein is used to simulate a laparoscopic 
medical procedure. In addition to the handle of a standard laparoscopic tool 18, the 
human/interface apparatus 12 may include a barrier 22 and a standard laparoscopic trocar 24 (or 

20 facsimile of a trocar). The barrier 22 is used to represent portion of the skin covering the body oi 
a patient. Trocar 24 is inserted into the body of the virtual patient to provide an entry an<: 
removal point from the body of the patient for the laparoscopic tool 18, and to allow th«: 
manipulation of the laparoscopic tool. Laparoscopic tools and trocars 24 are commercial!) 
available from sources such as U.S. Surgical of Connecticut. Barrier 22 and trocar 24 can b: 

25 omitted from apparatus 12 in other embodiments. Preferably, the laparoscopic tool 18 i: 
modified; in the preferred embodiment, the shaft is replaced by a linear axis member of thi 
present invention, as described below. In other embodiments, the end of the shaft of the too 
(such as any cutting edges) can be removed. The end of the laparoscopic tool 18 is not require: 
for the virtual reality simulation, and is removed to prevent any potential damage to persons o 

30 property. An apparatus 25 for interfacing mechanical input and output is shown within th< 
"body" of the patient in phantom lines. 

The laparoscopic tool 18 includes a handle or "grip" portion 26 and a shaft portion 2£ 
The shaft portion is an elongated mechanical object and, in particular, is an elongated cylindria 
object, described in greater detail below. In one embodiment, the present invention is conceme 
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with tracking the movement of the shaft portion 28 in three-dimensional space, where the 
movement has been constrained such that the shaft portion 28 has only three or four free degrees 
of motion. This is a good simulation of the real use of a laparoscopic tool 18 in that once it is 
inserted into a trocar 24 and through the gimbal apparatus 25, it is limited to about four degrees 
5 of freedom. More particularly, the shaft 28 is constrained at some point of along its length such 
that it can move with four degrees of freedom within the patient's body. 

While one embodiment of the present invention will be discussed with reference to the 
laparoscopic tool 18, it will be appreciated that a great number of other types of objects can be 
used with the method and apparatus of the present invention. In fact, the present invention can 
10 be used with any mechanical object where it is desirable to provide a human/computer interface 
with three to six degrees of freedom. Such objects may include endoscopic or other similar 
surgical tools used in medical procedures, catheters, hypodermic needles, wires, fiber optic 
bundles, styluses, joysticks, screw drivers, pool cues, etc. Some of these other objects are 
described in detail subsequently. 

15 The electronic interface 14 is a component of the human/computer interface apparatus 

12 and couples the apparatus 12 to the computer 16. More particularly, interface 14 is used in 
preferred .embodiments to couple the various actuators and sensors contained in apparatus 12 
(which actuators and sensors are described in detail below) to computer 16. A suitable interface 
14 is described in detail with reference to Figure 9. 

20 The electronic interface 14 is coupled to mechanical apparatus 25 of the apparatus 12 by 

a cable 30 and is coupled to the computer 16 by a cable 32. In other embodiments, signal can be 
sent to and from interface 14 and computer 16 by wireless transmission and reception. In some 
embodiments of the present invention, interface 14 serves solely as an input device for the 
computer 16. In other embodiments of the present invention, interface 14 serves solely as an 

25 output device for the computer 16. In preferred embodiments of the present invention, the 
interface 14 serves as an input/output (I/O) device for the computer 16. 

In Figure 2, a schematic diagram of mechanical apparatus 25 for providing mechanical 
input and output in accordance with the present invention is shown. Apparatus 25 includes a 
gimbal mechanism 38 and a linear axis member 40. A user object 44 is preferably coupled to 
30 linear axis member 40. 

Gimbal mechanism 38, in the described embodiment, provides support for apparatus 25 
on a grounded surface 56 (schematically shown as part of member 46). Gimbal mechanism 38 is 
preferably a five-member linkage that includes a ground member 46, extension members 48a and 
48b, and central members 50a and 50b. Ground member 46 is coupled to a base or surface 
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which provides stability for apparatus 25, Ground member 46 is shown in Figure 2 as two 
separate members coupled together through grounded surface 56. The members of gimbal 
mechanism 38 axe rotatably coupled to one another through the use of bearings or pivots, 
wherein extension member 48a is rotatably coupled to ground member 46 and can rotate about an 

5 axis A, central member 50a is rotatably coupled to extension member 48a and can rotate about a 
floating axis D, extension member 48b is rotatably coupled to ground member 46 and can rotate 
about axis B, central member 50b is rotatably coupled to extension member 48b and can rotate 
about floating axis E, and central member 50a is rotatably coupled to central member 50b at a 
center point P at the intersection of axes D and E. The axes D and E are "floating" in the sense 

10 that they arc not fixed in one position as are axes A and B. Axes A and B are substantially 
mutually perpendicular. As used herein, "substantially perpendicular" will mean that two objects 
or axis are exactly or almost perpendicular, i.e. at least within five degrees or ten degrees of 
perpendicular, or more preferably within less than one degree of perpendicular. Similarly, the 
term "substantially paraller will mean that two objects or axis are exactly or almost parallel, i.e. 

15 are at least within five or ten degrees of parallel, and are preferably within less than one degree of 
parallel. 

Gimbal mechanism 38 is formed as a five member closed chain. Each end of one 
member is coupled to the end of a another member. The five-member linkage is arranged such 
that extension member 48a, central member 50a, and central member 50b can be rotated about 
20 axis A in a first degree of freedom. The linkage is also arranged such that extension member 
48b, central member 50b, and central member 50a can be rotated about axis B in a second degree 
of freedom. 

Linear axis member 40 is preferably an elongated rod-like member which is coupled to 
central member 50a and central member 50b at the point of intersection P of axes A and B. As 

25 shown in Figure 1, linear axis member 40 can be used as shaft 28 of user object 44. In other 
embodiments, linear axis member 40 is coupled to a different object. Linear axis member 40 is 
coupled to gimbal mechanism 38 such that it extends out of the plane defined by axis A and axis 
B. Linear axis member 40 can be rotated about axis A by rotating extension member 48a, central 
member 50a, and central member 50b in a first revolute degree of freedom, shown as arrow line 

30 51. Member 40 can also be rotated about axis B by rotating extension member 50b and the two 
central members about axis B in a second revolute degree of freedom, shown by arrow line 52. 
Being also translatably coupled to the ends of central members 50a and 50b, linear axis member 
40 can be linearly moved along floating axis C, providing a third degree of freedom as shown by 
arrows 53. Axis C can, of course, be rotated about one or both axes A and B as member 40 is 

35 rotated about these axes. 
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Also preferably coupled to gimbal mechanism 38 and linear axis member 40 are 
transducers, such as sensors and actuators. Such transducers are preferably coupled at the link 
points between members of the apparatus and provide input to and output from an electrical 
system, such as computer 16. Transducers that can be used with the present invention are 
5 described in greater detail with respect to Figure 2. 

User object 44 is coupled to apparatus 25 and is preferably an interface object for a user 
to grasp or otherwise manipulate in three dimensional (3D) space. One preferred user object 44 
is the grip 26 of a laparoscopic tool 18, as shown in Figure 1. Shaft 28 of tool 18 can be 
implemented as part of linear axis member 40. Other examples of user objects are described in 
10 subsequent embodiments. User object 44 may be moved in all three degrees of freedom 
provided by gimbal mechanism 38 and linear axis member 40 and additional degrees of freedom 
as described below. As user object 44 is moved about axis A, floating axis D varies its position, 
and as user object 44 is moved about axis B t floating axis E varies its position. 

Figures 3 and 4 are perspective views of a specific embodiment of a mechanical apparatus 
15 25* for providing mechanical input and output to a computer system in accordance with the 
present invention. Figure 3 shows a front view of apparatus 25', and Figure 4 shows a rear 
view of the apparatus. Apparatus 25' includes a gimbal mechanism 38, a linear axis member 40, 
and transducers 42. A user object 44, shown in this embodiment as a laparoscopic instrument 
having a grip portion 26, is coupled to apparatus 25\ Apparatus 25' operates in substantially the 
20 same fashion as apparatus 25 described with reference to Figure 2. 

Gimbal mechanism 38 provides support for apparatus 25* on a grounded surface 56, 
such as a table top or similar surface. The members and joints ("bearings 0 ) of gimbal mechanism 
38 are preferably made of a lightweight, rigid, stiff metal, such as aluminum, but can also be 
made of other rigid materials such as other metals, plastic, etc. Gimbal mechanism 38 includes a 

25 ground member 46, capstan drive mechanisms 58, extension members 48a and 48b, central drive 
member 50a, and central link member 50b. Ground member 46 includes a base member 60 and 
vertical support members 62. Base member 60 is coupled to grounded surface 56 and provides 
two outer vertical surfaces 61 which are in a substantially perpendicular relation which each 
other. A vertical support member 62 is coupled to each of these outer surfaces of base member 

30 60 such that vertical members 62 are in a similar substantially 90-degrce relation with each other. 

A capstan drive mechanism 58 is preferably coupled to each vertical member 62. Capstan 
drive mechanisms 58 are included in gimbal mechanism 38 to provide mechanical advantage 
without introducing friction and backlash to the system. A capstan drum 59 of each capstan 
drive mechanism is rotatably coupled to a corresponding vertical support member 62 to form 



WO 96/22591 



PCT/US96/00701 



16 

axes of rotation A and B, which correspond to axes A and B as shown in Figure 1 . The capstan 
drive mechanisms 58 are described in greater detail with respect to Figure 5. 

Extension member 48a is rigidly coupled to capstan drum 59 and is rotated about axis A 
as capstan drum 59 is rotated. Likewise, extension member 48b is rigidly coupled to the other 
5 capstan drum 59 and can be rotated about axis B. Both extension members 48a and 48b are 
formed into a substantially 90-degree angle with a short end 49 coupled to capstan drum 59. 
Central drive member 50a is rotatably coupled to a long end 5 1 of extension member 48a and 
extends at a substantially parallel relation with axis B. Similarly, central link member 50b is 
rotatably coupled to the long end of extension member 48b and extends at a substantially parallel 
10 relation to axis A (as better viewed in Figure 4). Central drive member 50a and central link 
member 50b are rotatably coupled to each other at the center of rotation of the gimbal mechanism, 
which is the point of intersection P of axes A and B. Bearing 64 connects the two central 
members 50a and 50b together at the intersection point P. 

Gimbal mechanism 38 provides two degrees of freedom to an object positioned at or 
15 coupled to the center point P of rotation. An object at or coupled to point P can be rotated about 
axis A and B or have a combination of rotational movement about these axes. 

Linear axis member 40 is a cylindrical member that is preferably coupled to central 
members 50a and 50b at intersection point P. In alternate embodiments, linear axis member 40 
can be a non-cylindrical member having a cross-section of, for example, a square or other 

20 polygon. Member 40 is positioned through the center of bearing 64 and through holes in the 
central members 50a and 50b. The linear axis member can be linearly translated along axis C , 
providing a third degree of freedom to user object 44 coupled to the linear axis member. Linear 
axis member 40 can preferably be translated by a transducer 42 using a capstan drive mechanism 
similar to capstan drive mechanism 58. The translation of linear axis member 40 is described in 

25 greater detail with respect to Figure 6. 

Transducers 42 are preferably coupled to gimbal mechanism li to provide input and 
output signals between mechanical apparatus 25* and computer 16. In the described 
embodiment, transducers 42 include two grounded transducers 66a and 66b, central transducer 
68, and shaft transducer 70. The housing of grounded transducer 66a is preferably coupled to 

30 vertical support member 62 and preferably includes both an actuator for providing force in or 
otherwise influencing the first revolute degree of freedom about axis A and a sensor for 
measuring the position of object 44 in or otherwise influenced by the first degree of freedom 
about axis A, i.e., the transducer 66a is "associated with" or ' related to" the first degree ot 
freedom. A rotational shaft of actuator 66a is coupled to a pulley of capstan drive mechanism 58 

35 to transmit input and output along the first degree of freedom. The capstan drive mechanism 58 
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is described in greater detail with respect to Figure 5. Grounded transducer 66b preferably 
corresponds to grounded transducer 66a in function and operation. Transducer 66b is coupled to 
the other vertical support member 62 and is an actuator/sensor which influences or is influenced 
by the second re volute degree of freedom about axis B. 

5 Grounded transducers 66a and 66b are preferably bi-directional transducers which 

include sensors and actuators. The sensors are preferably relative optical encoders which 
provide signals to measure the angular rotation of a shaft of the transducer. The electrical outputs 
of the encoders are routed to computer interface 14 via buses 67a and 67b and are detailed with 
reference to Figure 9. Other types of sensors can also be used, such as potentiometers, etc. 

10 It should be noted that the present invention can utilize both absolute and relative 

sensors. An absolute sensor is one which the angle of the sensor is known in absolute terms, 
such as with an analog potentiometer. Relative sensors only provide relative angle information, 
and thus require some form of calibration step which provide a reference position for the relative 
angle information. The sensors described herein are primarily relative sensors. In consequence. 

15 there is an implied calibration step after system power-up wherein the sensor's shaft is placed in 
a known position within the apparatus 25' and a calibration signal is provided to the system to 
provide the reference position mentioned above. "All angles provided by the sensors are 
thereafter relative to that reference position. Such calibration methods are well known to those 
skilled in the art and, therefore, will not be discussed in any great detail herein. 

20 Transducers 66a and 66b also preferably include actuators which, in the described 

embodiment, are linear current control motors, such as DC servo motors. These motors 
preferably receive current signals to control the direction and torque (force output) that is 
produced on a shaft; the control signals for the motor are produced by computer interface 14 on 
control buses 67a and 67b and arc detailed with respect to Figure 9. The motors may include 

25 brakes which allow the rotation of the shaft to be halted in a short span of time. A suitable 
transducer for the present invention including both an optical encoder and current controlled 
motor is a 20 W basket wound servo motor manufactured by Maxon of Burlingame, California. 

In alternate embodiments, other types of motors can be used, such as a stepper motor 
controlled with pulse width modulation of an applied voltage, or pneumatic motors. However, 
30 the present invention is much more suited to the use of linear current controlled motors. This is 
because voltage pulse width modulation or stepper motor control involves the use of steps or 
pulses which can be felt as "noise" by the user. Such noise corrupts the virtual simulation. 
Linear current control is smoother and thus more appropriate for the present invention. 
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Passive actuators can also be used in transducers 66a, 66b and 68. Magnetic particle 
brakes or friction brakes can be used in addition to or instead of a motor to generate a passive 
resistance or friction in a degree of motion. An alternate preferred embodiment only including 
passive actuators may not be as realistic as an embodiment including motors; however, the 
5 passive actuators are typically safer for a user since the user does not have to fight generated 
forces. 

In other embodiments, all or some of transducers 42 can include only sensors to provide 
an apparatus without force feedback along designated degrees of freedom. Similarly, all or some 
of transducers 42 can be implemented as actuators without sensors to provide only force 
10 feedback. 

Central transducer 68 is coupled to central drive member 50a and preferably includes an 
actuator for providing force in the linear third degree of freedom along axis C and a sensor for 
measuring the position of object 44 along the third degree of freedom. The rotational shaft of 
central transducer 68 is coupled to a translation interface coupled to central drive member 50a 
15 which is described in greater detail with respect to Figure 6. In the described embodiment, 
central transducer 68 is an optical encoder and DC servo motor combination similar t6 the 
actuators 66a and 66b described above. 

The transducers 66a, 66b and 68 of the described embodiment are advantageously 
positioned to provide a very low amount of inertia to the user handling object 44. Transducer 

20 66a and transducer 66b are decoupled, meaning that the transducers are both directly coupled to 
ground member 46 which is coupled to ground surface 56, i.e. the ground surface carries the 
weight of the transducers, not the user handling object 44. The weights and inertia of the 
transducers 66a and 66b are thus substantially negligible to a user handling and moving objecn 
44. This provides a more realistic interface to a virtual reality system, since the computer can 

25 control the transducers to provide substantially all of the forces felt by the user in these degrees 
of motion. Apparatus 25* is a high bandwidth force feedback system, meaning that high 
frequency signals can be used to control transducers 42 and these high frequency signals will tx: 
applied to the user object with high precision, accuracy, and dependability. The user feels ven< 
little compliance or "mushiness" when handling object 44 due to the high bandwidth. Ir. 

30 contrast, in typical prior art arrangements of multi-degree of freedom interfaces, one actuatoi 
"rides" upon another actuator in a serial chain of links and actuators. This low bandwidtl: 
arrangement causes the user to feel the inertia of coupled actuators when manipulating an object. 

Central transducer 68 is positioned near the center of rotation of two revolute degrees o 
freedom. Though the transducer 68 is not grounded, its central position permits a minima 
35 inertial contribution to the mechanical apparatus 25* along the provided degrees of freedom. r 
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user manipulating object 44 thus will feel minimal internal effects from the weight of transducers 
66a, 66b and 68. 

Shaft transducer 70 preferably includes a sensor and is provided in the described 
embodiment to measure a fourth degree of freedom for object 44. Shaft transducer 70 is 
5 preferably positioned at the end of linear axis member 40 that is opposite to the object 44 and 
measures the rotational position of object 44 about axis C in the fourth degree of freedom, as 
indicated by arrow 72. Shaft transducer 70 is described in greater detail with respect to Figure 6 
and 6b. Preferably, shaft transducer 72 is implemented using an optical encoder similar to the 
encoders described above. A suitable input transducer for use in the present invention is an 

10 optical encoder model SI marketed by U.S. Digital of Vancouver, Washington. In the described 
embodiment, shaft transducer 70 only includes a sensor and not an actuator. This is because for 
typical medical procedures, which is one intended application for the embodiment shown in 
Figures 3 and 4, rotational force feedback to a user about axis C is typically not required to 
simulate actual operating conditions. However, in alternate embodiments, an actuator such as a 

15 motor can be included in shaft transducer 70 similar to transducers 66a, 66b, and 68. 

Object 44 is shown in Figures 3 and 4 as a grip portion 26 of a laparoscopic tool similar " 
to the tool shown in Figure 1. Shaft portion 28 is implemented as linear axis member 40. A user 
can move the laparoscopic tool about axes A and B, and can translate the tool along axis C and 
rotate the tool about axis C. The movements in these four degrees of freedom will be sensed and 
20 tracked by computer system 16. Forces can be applied preferably in the first three degrees of 
freedom by the computer system to simulate the tool impacting a portion of subject body, 
experiencing resistance moving through tissues, etc. 

Optionally, additional transducers can be added to apparatus 25' to provide additional 
degrees of freedom for object 44. For example, a transducer can be added to grip 26 of 
25 laparoscopic tool 18 to sense when the user moves the two portions 26a and 26b relative to each 
other to simulate extending the cutting blade of the tool. Such a laparoscopic tool sensor is 
described in U.S. Patent Application Serial No. 08/275,120. filed July 14. 1994 and entitled 
"Method and Apparatus for Providing Mechanical I/O for Computer Systems" assigned to the 
assignee of the present invention and incorporated herein by reference in its entirety. 

30 Figure 5 is a perspective view of a capstan drive mechanism 58 shown in some detail. 

As an example, the drive mechanism 58 coupled to extension aim 48b is shown; the other 
capstan drive 58 coupled to extension arm 48a is substantially similar to the mechanism presented 
here. Capstan drive mechanism 58 includes capstan drum 59, capstan pulley 76, and stop 78. 
Capstan drum 59 is preferably a wedge-shaped member having leg portion 82 and a curved 

35 portion 84. Other shapes of member 59 can also be used. Leg portion 82 is pivotally coupled to 
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vertical support member 62 at axis B (or axis A for the opposing capstan drive mechanism). 
Extension member 48b is rigidly coupled to leg portion 82 such that when capstan drum 59 is 
rotated about axis B, extension member 48b is also rotated and maintains the position relative to 
leg portion 82 as shown in Figure 5. Curved portion 84 couples the two ends of leg portion 82 
5 together and is preferably formed in an arc centered about axis B. Curved portion 84 is 
preferably positioned such that its bottom edge 86 is about 0.030 inches above pulley 76. 

Cable 80 is preferably a thin metal cable connected to curved portion 84 of the capstan 
drum. Other types of durable cables, cords, wire, etc. can be used as well. Cable 80 is attached 
at a first end to curved portion 84 near an end of leg portion 82 and is drawn taudy against the 

10 outer surface 86 of curved portion 84. Cable 80 is wrapped around pulley 76 a number of times 
and is then again drawn tautiy against outer surface 86. The second end of cable 80 is firmly 
attached to the other end of curved portion 84 near the opposite leg of leg portion 82. The cable 
transmits rotational force from pulley 76 to the capstan drum 59, causing capstan drum 59 to 
rotate about axis B as explained below. The cable also transmits rotational force from drum 59 to 

15 the pulley and transducer 66b. The tension in cable 80 should be at a level so that negligible 
backlash or play occurs between capstan drum 59 and pulley 76. Preferably, the tension of cable 
80 can be adjusted by pulling more (or less) cable length through an end of curved portion 84. 
Caps 81 on the ends of curved portion 84 can be used to easily tighten cable 80. Each cap 81 is 
preferably tightly coupled to cable 80 and includes a pivot and tightening screw which allow the 

20 cap to move in a direction indicated by arrow 83 to tighten cable 80. 

Capstan pulley 76 is a threaded metal cylinder which transfers rotational force from 
transducer 66b to capstan drum 59 and from capstan drum 59 to transducer 66b. Pulley 76 is 
rotationally coupled to vertical support member 62 by a shaft 88 (shown in Figure 5a) positioned 
through a bore of vertical member 62 and rigidly attached to pulley 76. Transducer 66b is 

25 coupled to pulley 76 by shaft 88 through vertical support member 62. Rotational force is applied 
from transducer 66b to pulley 76 when the actuator of transducer 66b rotates the shaft. The 
pulley, in tum, transmits the rotational force to cable 80 and thus forces capstan drum 59 to rotate 
in a direction about axis B. Extension member 48b rotates with capstan drum 59, thus causing 
force along the second degree of freedom for object 44. Note that pulley 76, capstan drum 59 

30 and extension member 48b will only actually rotate if the user is not applying the same amount or 
a greater amount of rotational force to object 44 in the opposite direction to cancel the rotational 
movement In any event, the user will feel the rotational force along the second degree of 
freedom in object 44 as force feedback. 

The capstan mechanism 58 provides a mechanical advantage to apparatus 25' so that the: 
35 force output of the actuators can be increased. The ratio of the diameter of pulley 76 to the 
diameter of capstan drum 59 (i.e. double the distance from axis B to the bottom edge 86 oi 



WO 96/22591 



PCT/US96/00701 



21 

capstan drum 59) dictates the amount of mechanical advantage, similar to a gear system. In the 
preferred embodiment, the ratio of drum to pulley is equal to 15:1, although other ratios can be 
used in other embodiments. 

Similarly, when the user moves object 44 in the second degree of freedom, extension 
5 member 48b rotates about axis B and rotates capstan drum 59 about axis B as well. This 
movement causes cable 80 to move, which transmits the rotational force to pulley 76. Pulley 76 
rotates and causes shaft 88 to rotate, and the direction and magnitude of the movement is detected 
by the sensor of transducer 66b. A similar process occurs along the first degree of freedom for 
the other capstan drive mechanism 58. As described above with respect to the actuators, the 
10 capstan drive mechanism provides a mechanical advantage to amplify the sensor resolution by a 
ratio of drum 59 to pulley 76 ( 15: 1 in the preferred embodiment). 

Stop 78 is rigidly coupled to vertical support member 62 a few millimeters above curved 
portion 84 of capstan drum 59. Stop 78 is used to prevent capstan drum 59 from moving 
beyond a designated angular limit. Thus, drum 59 is constrained to movement within a range 
15 defined by the arc length between the ends of leg portion 82. This constrained movement, in 
turn, constrains the movement of object 44 in the first two degrees of freedom. In the described " 
embodiment, stop 78 is a cylindrical member inserted into a threaded bore in vertical support 
member 62. 

Figure 5a is a side elevational view of capstan mechanism 58 as shown in Figure 5. 

20 Cable 80 is shown routed along the bottom side 86 of curved portion 84 of capstan drum 59. 
Cable 80 is preferably wrapped around pulley 76 so that the cable is positioned between threads 
90, i.e., the cable is guided by the threads as shown in greater detail in Figure 5b. As pulley 76 
is rotated by transducer 66b or by the manipulations of the user, the portion of cable 80 wrapped 
around the pulley travels closer to or further from vertical support member 62, depending on the 

25 direction that pulley 76 rotates. For example, if pulley 76 is rotated counterclockwise (when 
viewing the pulley as in Figure 5), then cable 80 moves toward vertical support member 62 as 
shown by arrow 92. Capstan drum 59 also rotates clockwise as shown by arrow 94. The 
threads of pulley 76 are used mainly to provide cable 80 with a better grip on pulley 76. In 
alternate embodiments, pulley 76 includes no threads, and the high tension in cable 80 allows 

30 cable 80 to grip pulley 76. 

Capstan drive mechanism 58 is advantageously used in the present invention to provide 
transmission of forces and mechanical advantage between transducers 66a and 66b and object 44 
without introducing substantial compliance, friction, or backlash to the system. A capstan drive 
provides increased stiffness, so that forces are transmitted with negligible stretch and 
35 compression of the components. The amount of friction is also reduced with a capstan drive 
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mechanism so thai substantially "noiseless" tactile signals can be provided to the user. In 
addition, the amount of backlash contributed by a capstan drive is also negligible. "Backlash" is 
the amount of play that occurs between two coupled rotating objects in a gear or pulley system. 
Two gears, belts, or other types of drive mechanisms could also be used in place of capstan drive 
mechanism 58 in alternate embodiments to transmit forces between transducer 66a and extension 
member 48b. However, gears and the like typically introduce some backlash in the system. In 
addition, a user might be able to feel the interlocking and grinding of gear teeth during rotation of 
gears when manipulating object 44; the rotation in a capstan drive mechanism is much less 
noticeable. 

Figure 6 is a perspective view of central drive member 50a and linear axis member 40 
shown in some detail. Central drive member 50a is shown in a partial cutaway view to expose 
the interior of member 50a. Central transducer 68 is coupled to one side of central drive member 
50a. In the described embodiment, a capstan drive mechanism is used to transmit forces between 
transducer 68 and linear axis member 40 along the third degree of freedom. A rotatable shaft 9& 
of transducer 68 extends through a bore in the side wall of central drive member 50a and i:; 
coupled to a capstan pulley 100. Pulley 100 is described in greater detail below with respect tc 
Figure 6a. 

Linear axis member 40 preferably includes an exterior sleeve 91 and an interior shaft 92 
(described with reference to Figure 6b, below). Exterior sleeve 91 is preferably a partially 
cylindrical member having a flat 41 provided along its length. Flat 41 prevents sleeve 91 fron: 
rotating about axis C in the fourth degree of freedom described above. Linear axis member 40 i: 
provided with a cable 99 which is secured on each end of member 40 by tension caps 101 
Cable 99 preferably runs down a majority of the length of exterior sleeve 91 on the surface of fla 
41 and can be tightened, for example, by releasing a screw 97, pulling an end of cable 99 unti 
the desired tension is achieved, and tightening screw 97. Similarly to the cable of the capstai 
mechanism described with reference to Figure 5, cable 99 should have a relatively high tension. 

As shown in Figure 6a, cable 99 is wrapped a number of times around pulley 100 so th^ 
forces can be transmitted between pulley 100 and linear axis member 40. Pulley 100 preferabl 
includes a central axle portion 103 and end lip portions 105. Exterior sleeve 91 is preferabl 
positioned such that flat 41 of the sleeve is touching or is very close to lip portions 105 on bot 
sides of axle portion 103. The cable 99 portion around pulley 100 is wrapped around centn 
axle portion 103 and moves along portion 103 towards and away from shaft 98 as the pulley i 
rotated clockwise and counterclockwise, respectively. The diameter of axle portion 103 : 
smaller than lip portion 105, providing space between the pulley 100 and flat 41 where cable 9 
is attached and allowing free movement of the cable. Pulley 100 preferably does not incluc 
threads, unlike pulley 76, since the tension in cable 99 allows the cable to grip pulley 100 tightl; 
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In other embodiments, pulley 100 can be a threaded or unthreaded cylinder similar to capstan 
pulley 76 described with reference to Figure 5, 

Using the capstan drive mechanism, transducer 68 can translate linear axis member 40 
along axis C when the pulley is rotated by the actuator of transducer 68. Likewise, when linear 
5 axis member 40 is translated along axis C by the user manipulating object 44, pulley 100 and 
shaft 98 are rotated; this rotation is detected by the sensor of transducer 68. The capstan drive 
mechanism provides low friction and smooth, rigid operation for precise movement of linear axis 
member 40 and accurate position measurement of the member 40. 

Other drive mechanisms can also be used to transmit forces to linear axis member and 
10 receive positional information from member 40 along axis C. For example, a drive wheel made 
of a rubber- like material or other frictional material can be positioned on shaft 98 to contact linear 
axis member 40 along the edge of the wheel. The wheel can cause forces along member 40 from 
the friction between wheel and linear axis member. Such a drive wheel mechanism is disclosed 
in the abovementioned Application Serial No. 08/275,120 as well as in U.S. Patent Application 
15 Serial No. 08/344,148, filed November 23, 1994 and entitled "Method and Apparatus for 
Providing Mechanical I/O for Computer Systems Interfaced with Elongated Flexible Objects" 
assigned to the assignee of the present invention and incorporated herein by reference in its 
entirety. Linear axis member 40 can also be a single shaft in alternate embodiments instead of a 
dual part sleeve and shaft. 

20 Referring to the cross sectional side view of member 40 and transducer 70 shown in 

Figure 6b, interior shaft 93 is positioned inside hollow exterior sleeve 91 and is rotatably coupled 
to sleeve 91. A first end 107 of shaft 93 preferably extends beyond sleeve 91 and is coupled to 
object 44. When object 44 is rotated about axis C, shaft 93 is also rotated about axis C in the 
fourth degree of freedom within sleeve 91. Shaft 93 is translated along axis C in the third degree 

25 of freedom when sleeve 91 is translated. Alternatively, interior shaft 93 can be coupled to a shaft 
of object 44 within exterior sleeve 91. For example, a short portion of shaft 28 of laparoscopic 
tool 18, as shown in Figure 1, can extend into sleeve 91 and be coupled to shaft 93 within the 
sleeve, or shaft 28 can extend all the way to transducer 70 and functionally be used as shaft 93. 

Shaft 93 is coupled at its second end 109 to transducer 70, which, in the preferred 
30 embodiment, is an optical encoder sensor. The housing 1 1 1 of transducer 70 is rigidly coupled 
to exterior sleeve 91 by a cap 115, and a shaft 1 1 3 of transducer 70 is coupled to interior shaft 93 
so that transducer 70 can measure the rotational position of shaft 93 and object 44. In alternate 
embodiments, an actuator can also be included in transducer 70 to provide rotational forces about 
axis C to shaft 93. 
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Figure 7 is a perspective view of an alternate embodiment of the mechanical apparatus 
25" and user object 44 of the present invention. Mechanical apparatus 25" shown in Figure 7 
operates substantially the same as apparatus 25' shown in Figures 3 and 4. User object 44, 
however, is a stylus 102 which the user can grasp and move in six degrees of freedom. By 
5 "grasp", it is meant that users may releasably engage a grip portion of the object in some fashion, 
such as by hand, with their fingertips, or even orally in the case of handicapped persons. Stylus 
102 can be sensed and force can be applied in various degrees of freedom by a computer system 
and interface such as computer 16 and interface 14 of Figure I. Stylus 102 can be used in virtual 
reality simulations in which the user can move the stylus in 3D space to point to objects, write 

10 words, drawings, or other images, etc. For example, a user can view a virtual environment 
generated on a computer screen or in 3D goggles. A virtual stylus can be presented in a virtual 
hand of the user. The computer system tracks the position of the stylus with sensors as the user 
moves it. The computer system also provides force feedback to the stylus when the user moves 
the stylus against a virtual desk top, writes on a virtual pad of paper, etc. It thus appears and 

1 5 feels to the user that the stylus is contacting a real surface. 

Stylus 102 preferably is coupled to a floating gimbal mechanism 104 which provides two 
degrees of freedom in addition to the four degrees of freedom provided by apparatus 25' 
described with reference to Figures 3 and 4. Floating gimbal mechanism 104 includes a Li- 
shaped member 106 which is rotatably coupled to an axis member 108 by a shaft 109 so that U- 

20 shaped member 106 can rotate about axis F. Axis member 108 is rigidly coupled to linear axis 
member 40. In addition, the housing of a transducer 1 10 is coupled to U-shaped member 106 
and a shaft of transducer 110 is coupled to shaft 109. Shaft 109 is preferably locked into 
position within axis member 108 so that as U-shaped member 106 is rotated, shaft 109 does not 
rotate. Transducer 1 10 is preferably a sensor, such as an optical encoder as described above 

25 with reference to transducer 70, which measures the rotation of U-shaped member 106 about 
axis F in a fifth degree of freedom and provides electrical signals indicating such movement to 
interface 14. 

Stylus 102 is preferably rotatably coupled to U-shaped member 106 by a shaft (not 
shown) extending through the U-shaped member. This shaft is coupled to a shaft of transducer 
30 1 12, the housing of which is coupled to U-shaped member 106 as shown. Transducer 112 i<; 
preferably a sensor, such as an optical encoder as described above, which measures the rotation 
of stylus 102 about the lengthwise axis G of the stylus in a sixth degree of freedom. 

In the described embodiment of Figure 7, six degrees of freedom of stylus 102 an: 
sensed. Thus, both the position (x, y, z coordinates) and the orientation (roll, pitch, yaw) of the 
35 stylus can be detected by computer 16 to provide a highly realistic simulation. Other mechanism; 
besides the floating gimbal mechanism 104 can be used to provide the fifth and sixth degrees ol 
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freedom. In addition, forces can be applied in three degrees of freedom for stylus 102 to provide 
3D force feedback. In alternate embodiments, actuators can also be included in transducers 70, 
110, and 112. However, actuators are preferably not included for the fourth, fifth, and sixth 
degrees of freedom in the described embodiment, since actuators are typically heavier than 
5 sensors and, when positioned at the locations of transducers 70, 100, and 112, would create 
more inertia in the system. In addition, the force feedback for the designated three degrees of 
freedom allows impacts and resistance to be simulated, which is typically adequate in many 
virtual reality applications. Force feedback in the fourth, fifth, and sixth degrees of freedom 
would allow torques on stylus 102 to be simulated as well, which may or may not be useful in a 
10 simulation. 

Figure 8 is a perspective view of a second alternate embodiment of the mechanical 
apparatus 25'" and user object 44 of the present invention. Mechanical apparatus 25"* shown 
in Figure 8 operates substantially the same as apparatus 25* shown in Figures 3 and 4. User 
object 44, however, is a joystick 1 12 which the user can preferably move in two degrees of 

15 freedom. Joystick 1 12 can be sensed and force can be applied in both degrees of freedom by a 
computer system and interface similar to computer system 16 and interface 14 of Figure 1 . In the 
described embodiment, joystick 112 is coupled to cylindrical fastener 64 so that the user can 
move the joystick in the two degrees of freedom provided by gimbal mechanism 38 as described 
above. Linear axis member 40 is not typically included in the embodiment of Figure 8, since a 

20 joystick is not usually translated along an axis C. However, in alternate embodiments, joystick 
112 can be coupled to linear axis member 40 similarly to stylus 102 as shown in Figure 7 to 
provide a third degree of freedom. In yet other embodiments, linear axis member 40 can rotate 
about axis C and transducer 70 can be coupled to apparatus 25' M to provide a fourth degree of 
freedom. Finally, in other embodiments, a floating gimbal mechanism as shown in Figure 7, or 

25 a different mechanism, can be added to the joystick to allow a full six degrees of freedom. 

Joystick 1 12 can be used in virtual reality simulations in which the user can move the 
joystick to move a vehicle, ooint to objects, control a mechanism* etc. For example, a user can 
view a virtual environment generated on a computer screen or in 3D goggles in which joystick 
112 controls an aircraft. The computer system tracks the position of the joystick as the user 
30 moves it around with sensors and updates the virtual reality display accordingly to make the 
aircraft move in the indicated direction, etc. The computer system also provides force feedback 
to the joystick, for example, when the aircraft is banking or accelerating in a turn or in other 
situations where the user may experience forces on the joystick or find it more difficult to steer 
the aircraft. 

35 Figure 9 is a schematic view of a computer 16 and an interface circuit 120 used in 

interface 14 to send and receive signals from mechanical apparatus 25. Circuit 120 includes 
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computer 16, interface card 120, DAC 122, power amplifier circuit 124, digital sensors 128, and 
sensor interface 130. Optionally included are analog sensors 132 instead of or in addition to 
digital sensors 128, and ADC 134. In this embodiment, the interface 14 between computer 16 
and mechanical apparatus 25 as shown in Figure 1 can be considered functionally equivalent to 
5 the interface circuits enclosed within the dashed line in Figure 14. Other types of interfaces 14 
can also be used. For example, an electronic interface 14 is described in U.S. Patent Application 
Serial No. 08/092,974, filed July 16, 1993 and entitled "3-D Mechanical Mouse" assigned to the 
assignee of the present invention and incorporated herein by reference in its entirety. The 
electronic interface described therein was designed for the Immersion PROBE™ 3-D mechanical 
10 mouse and has six channels corresponding to the six degrees of freedom of the Immersion 
PROBE. 

Interface card 120 is preferably a card which can fit into an interface slot of computer 1 6. 
For example, if computer 16 is an IBM AT compatible computer, interface card 14 can be 
implemented as an ISA or other well-known standard interface card which plugs into the 
15 motherboard of the computer and provides input and output ports connected to the main data bus 
of the computer. 

Digital to analog converter (DAC) 122 is coupled to interface card 120 and receives a 
digital signal from computer 16. DAC 122 converts the digital signal to analog voltages which 
are then sent to power amplifier circuit 124. A DAC circuit suitable for use with the present 

20 invention is described with reference to Figure 10. Power amplifier circuit 124 receives an 
analog low-power control voltage from DAC 122 and amplifies the voltage to control actuators. 
126. Power amplifier circuit 124 is described in greater detail with reference to Figure 11. 
Actuators 126 are preferably DC servo motors incorporated into the transducers 66a, 66b, and 
68, and any additional actuators, as described with reference to the embodiments shown in 

25 Figures 3, 7, and 8 for providing force feedback to a user manipulating object 44 coupled to 
mechanical apparatus 25. 

Digital sensors 128 provide signals to computer 16 relating the position of the user object 
44 in 3D space. In the preferred embodiments described above, sensors 128 are relative optica! 
encoders, which are electro-optical devices that respond to a shaft's rotation by producing twe 

30 phase-related signals. In the described embodiment, sensor interface circuit 130, which v. 
preferably a single chip, receives the signals from digital sensors 128 and converts the twe 
signals from each sensor into another pair of clock signals, which drive a bi-directional binan 
counter. The output of the binary counter is received by computer 16 as a binary numbe 
representing the angular position of the encoded shaft. Such circuits, or equivalent circuits, an 

35 well known to those skilled in the art; for example, the Quadrature Chip from Hewlett Packard 
California performs the functions described above. 
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Analog sensors 132 can be included instead of digital sensors 128 for all or some of the 
transducers of the present invention. For example, a strain gauge can be connected to stylus 130 
of Figure 7 to measure forces. Analog sensors 132 provide an analog signal representative of the 
position of the user object in a particular degree of motion. Analog to digital converter (ADC) 
5 134 converts the analog signal to a digital signal that is received and interpreted by computer 16, 
as is well known to those skilled in the art. 

Figure 10 is a schematic view of a DAC circuit 122 of Figure 9 suitable for converting an 
input digital signal to an analog voltage that is output to power amplifier circuit 124. In the 
described embodiment, circuit 122 includes a parallel DAC 136, such as the DAC1220 

10 manufactured by National Semiconductor, which is designed to operate with an external generic 
op amp 138. Op amp 1 38, for example, outputs a signal from zero to -5 volts proportional to the 
binary number at its input. Op amp 140 is an inverting summing amplifier that converts the 
output voltage to a symmetrical bipolar range. Op amp 140 produces an output signal between - 
2.5 V and +2.5 V by inverting the output of op amp 138 and subtracting 2.5 volts from that 

15 output; this output signal is suitable for power amplification in amplification circuit 124. As an 
example, Rl = 200 kfl and R2 = 400 led Of course, circuit 122 is intended as one example of 
many possible circuits that can be used to convert a digital signal to a desired analog signal. 

Figure 11 is a schematic view of a power amplifier circuit 124 suitable for use in the 
interface circuit 14 shown in Figure 9. Power amplifier circuit receives a low power control 

20 voltage from DAC circuit 122 to control high-power, cunenKontrolled servo motor 126. The 
input control voltage controls a transconductance stage composed of amplifier 142 and several 
resistors. The transconductance stage produces an output current proportional to the input 
voltage to drive motor 126 while drawing very little current from the input voltage source. The 
second amplifier stage, including amplifier 144, resistors, and a capacitor C, provides additional 

25 current capacity by enhancing the voltage swing of the second terminal 147 of motor 146. As 
example values for circuit 124, R = 10 kQ , R2 = 500 Q, R3 = 9.75 kfl, and R4 = 1 n. Of 
course, circuit 124 is intended as one example of many possible circuits that c*n be used to 
amplify voltages to drive actuators 126. 

Figure 12a is a schematic diagram of a transducer system 200 suitable for use with the 
30 present invention. Transducer system 200 is ideally suited for an interface system in which 
passive actuators, instead of active actuators, are implemented. As shown in Figure 12a, 
transducer system 200 is applied to a mechanism having one degree of freedom, as shown by 
arrows 201. Embodiments in which system 200 is applied to systems having additional degrees 
of freedom are described subsequently. Transducer system 200 includes an actuator 202, an 
35 actuator shaft 204, a non-rigidly attached coupling 206, a coupling shaft 208, a sensor 210, and 
an object 44. 
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Actuator 202 transmits a force to object 44 and is preferably grounded, as shown by 
symbol 203. Actuator 202 is rigidly coupled to an actuator shaft 204 which extends from 
actuator 202 to non-rigidly attached coupling 206. Actuator 202 provides rotational forces, 
shown by arrows 212, on actuator shaft 204. In the preferred embodiment, actuator 202 is a 

5 passive actuator which can apply a resistive or factional force (i.e., drag) to shaft 204 in the 
directions of arrow 2 12 but cannot provide an active force to shaft 204 (i.e., actuator 202 cannot 
cause shaft 204 to rotate). Thus, an external rotational force, such as a force generated by a user, 
is applied to shaft 204, and passive actuator 202 provides resistive forces to that external 
rotational force. Preferred passive actuators include rotary magnetic brakes, and, in particular, 

10 magnetic particle brakes, which are low cost and power-efficient devices. Suitable magnetic 
particle brakes can be obtained from Force Limited, Inc. of Santa Monica, California. 

Passive actuators can provide realistic force feedback to a user operating an interface 
apparatus in a simulated environment. Passive actuators impose a resistance to the motion of an 
object 44 manipulated by the user. Thus, a user who manipulates an interface having passive 
1 5 actuators will feel forces only when he or she actually moves an object of the interface. 

Passive actuators 202 provide several advantages when compared to active actuators. A 
substantially lower current is required to drive passive actuators than active actuators. This 
allows a less expensive power supply to drive a passive actuator system, and also allows a force 
feedback mechanism to be smaller and more lightweight due to the smaller power supply. In 

20 addition, passive actuators require substantially slower control signals to operate effectively in a 
simulation environment than do active actuators such as motors. This is significant if the 
controller of an interface mechanism is a computer system that includes only a standard, low- 
speed input/output port, such as a serial port. Serial ports are quite common to persona] 
computers but do not communicate quickly enough to perform real-time, stable control of most 

25 active actuators. When using a controller with slower control signals, passive actuators can 
provide stable force feedback to the user. Another advantage of passive actuators, as explained 
above, is that passive actuators do not generate forces on the interface and the user and are thus 
more safe for the user. 

Coupling 206 is coupled to actuator shaft 204. Acutator 202, actuator shaft 204, and 
30 coupling 206 can be considered to be an "actuator assembly" or, in a passive actuating system, ;) 
"braking mechanism." Coupling 206 is preferably not rigidly coupled to actuator shaft 204 and 
thus allows an amount (magnitude) of "play" between actuator shaft 204 and coupling 206. The 
term "play," as used herein, refers to an amount of free movement or "looseness" between 2 
transducer and the object transduced, so that, for instance, the object can be moved a shor 
35 distance by externally-applied forces without being affected by forces applied to the object by a: 
actuator. In the preferred embodiment, the user can move the object a short distance withou 
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fighting the drag induced by a passive actuator such as a brake. For example, actuator 202 can 
apply a resistive or frictional force to actuator shaft 204 so that actuator shaft 204 is locked in 
place, even when force is applied to the shaft Coupling 206, however, can still be freely rotated 
by an additional distance in either rotational direction due to the play between coupling 206 and 

5 shaft 204. This play is intentional for purposes that will be described below, and is thus referred 
to as a "desired" amount of play. Once coupling 206 is rotated to the limit of the allowed play, it 
either forces shaft 204 to rotate with it further, or, if actuator 202 is holding (i.e., locking) shaft 
204, the coupling cannot be further rotated in that rotational direction. The amount of desired 
play between actuator 202 and object 44 greatly depends on the resolution of the sensor 210 

10 being used, and is described in greater detail below. Examples of types of play include rotary 
backlash, such as occurs in gear systems as described in the above embodiments, and 
compliance or torsion flex, which can occur with flexible, rotational and non-rotational members. 
Embodiments including these forms of play are described in greater detail below with reference 
to Figures 13 and 16, respectively. 

1 5 Coupling shaft 208 is rigidly coupled to coupling 206 and extends to sensor 210. Sensor 

210 is preferably rigidly coupled to coupling shaft 208 so as to detect rotational movement of m 
shaft 208 and object 44 about axis H. Sensor 210 preferably provides a electrical signal 
indicating the rotational position of shaft 208 and is preferably grounded as indicated by symbol 
211. In the described embodiment, sensor 210 is a digital optical encoder, similar to the 

20 encoders described in the above embodiments of Figures 1-11. In alternate embodiments, sensor 
210 can be separated from object 44, coupling shaft 208, and coupling 206. For example, a 
sensor having an emitter and detector of electromagnetic energy might be disconnected from the 
rest of transducer system 200 yet be able to detect the rotational position of object 44 using a 
beam of electromagnetic energy, such as infrared light. Similarly, a magnetic sensor could detect 

25 the position of object 44 while being uncoupled to shaft 208 or object 44. The operation of such 
sensors are well-known to those skilled in the an. 

Sensor 210 has a sensing resolution, which is the smallest change in rotational position 
of coupling shaft 208 that the sensor can detect. For example, an optical encoder of the 
described embodiment may be able to detect on the order of about 3600 equally-spaced "pulses" 

30 (described below) per revolution of shaft 208, which is about 10 detected pulses per degree of 
rotational movement Thus, the sensing resolution of this sensor is about 1/10 degree in this 
example. Since it is desired to detect the desired play between actuator 202 and object 44 (as 
described below), this desired play should not be less than the sensing resolution of sensor 210 
(e.g., 1/10 degree). Preferably, the desired play between actuator and object would be at least 

35 1/5 degree in this example, since the encoder could then detect two pulses of movement, which 
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would provide a more reliable measurement and allow the direction of the movement to be more 
easily determined. 

Sensor 210 should also be as rigidly coupled to shaft 208 as possible so that the sensor 
can detect the desired play of shaft 208 and object 44. Any play between sensor 210 and object 
5 44 should be minimized so that such play does not adversely affect the sensor's measurements. 
Typically, any inherent play between sensor 210 and object 44 should be less than the sensing 
resolution of the sensor, and preferably at least an order of magnitude less than the sensing 
resolution. Thus, in the example above, the play between sensor and object should be less than 
1/10 degree and preferably less than 1/100 degree. Use of steel or other rigid materials for shaft 

10 208 and other components, which is preferred, can allow the play between sensor 210 and object 
44 to be made practically negligible for purposes of the present invention. As referred to herein, 
a sensor that is "rigidly" coupled to a member has a play less than the sensing resolution of the 
sensor (preferably a negligible amount). The play between actuator 202 and object 44 is 
described in greater detail below. A suitable encoder to be used for sensor 210 is the "Softpot" 

15 from U.S. Digital of Vacouver, Washington. 

Object 44 is rigidly coupled to coupling shaft 208. Object 44 can take a variety of fortns, 
as described in previous embodiments, and-can be directly coupled to coupling shaft 208 or can 
be coupled through other intermediate members to shaft 208 . In Figure 12a, object 44 is 
coupled to shaft 208 between coupling 206 and sensor 210. Thus, as object 44 is rotated about 

20 axis H, shaft 208 is also rotated about axis H and sensor 210 detects the magnitude and direction 
of the rotation of object 44. Alternatively, object 44 can be coupled directly to coupling 206. 
Coupling 206 and/or shafts 204 and 208 can be considered a "play mechanism" for providing the 
desired play between actuator 202 and object 44. Certain suitable objects 44 include a joystick, 
medical instrument (catheter, laparoscope, etc.), a steering wheel (e.g. having one degree of 

25 freedom), a pool cue, etc. 

As stated above, transducer system 200 is ideally suited for mechanical systems that 
include low-cost elements such as passive actuators. If a controlling computer system, such as 
computer system 16, is to provide accurate force feedback to an object being held and moved by 
a user, the computer system should be able to detect the direction that the user is moving the 

30 object even when the passive actuators are being applied to the object at maximum force to lock 
the object in place. However, this can be difficult when using passive actuators, because passive 
rotary actuators provide a resistive force or friction to motion in both rotational directions about 
an axis. Thus, when force from an actuator prevents movement of an object in one direction, it 
also prevents movement in the opposite direction. This typically does not allow the computer to 

35 sense movement of the object in the opposite direction, unless the user provides a greater force 
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than the actuator's resistive force and overcomes the actuator's force (i.e., overpowers the 
actuator). 

For example, object 44 is a one-degree -of-frcedom joystick used for moving a video 
cursor that moves in the direction indicated by the joystick on a video screen. The user moves 

5 the cursor into a virtual (computer generated) "wall", which blocks the motion of the cursor in 
one direction. The controlling computer system also applies force feedback to the joystick by 
activating passive magnetic particle brakes to prevent the user from moving the joystick in the 
direction of the wall, thus simulating the surface of the wall. If sensor 210 is rigidly coupled to 
actuator shaft 204, a problem occurs if the user wishes to move the joystick in the opposite 

10 direction to the wall. Since the brakes have locked the joystick in both directions, the computer 
cannot detect when the user switches the joystick's direction unless the user overpowers the 
passive brakes. Thus, to the user, the cursor feels like it is "stuck" to the wall. 

Applicant's introduced ("desired") play between object 44 and actuator 202 solves this 
problem effectively and inexpensively. The play allows the joystick or other connected object to 

15 be moved slightly in the opposite direction even when the brakes are applied with maximum 
friction to the joystick. The sensor, being rigidly attached to the joystick, is not locked by the " 
actuator and detects the changenn direction. The sensor relays the movement to the computer, 
which deactivates the brakes to allow the joystick to be moved freely in the opposite direction. If 
the user should move the cursor into the wall again, the brakes would be similarly activated. A 

20 method for controlling actuator 202 in such a virtual reality environment is described with 
reference to Figure 22. 

Active actuators, such as the DC motors described in the above embodiments of Figures 
3-8 or other types of motors, can also be used with transducer system 200. Many active 
actuators, however, can apply force in one selected direction in a degree of freedom, so that the 
25 deliberately-introduced play would not be necessary when using such actuators. 

In alternate embodiments, linear play can be implemented instead of rotary play. The 
preferred embodiments of Figures 12a and 12b (described below) implement play among 
rotational components, such as a rotary actuator and sensor. However, compliance or backlash 
can also be implemented between linearly moving (i.e., translatable) components. For example, 
30 a small amount of space can be provided between interlocked translatable components to provide 
play in accordance with the present invention. An actuator and sensor for transducing linear 
movement, which are well-known to those skilled in the art, can be used in such an embodiment. 

Other devices or mechanisms besides the use of play can be used in other embodiments to 
detect the direction of motion of object 44 while passive actuators are holding the object in place. 
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For example, force sensors can be coupled to the object to measure the force applied to the object 
by the user along desired degrees of freedom. A force sensor can detect if a user is applying a 
force, for example, towards the virtual wall or away from the virtual wall, and the computer can 
activate or deactivate the passive actuators accordingly. Deliberately-introduced play between 
5 object and actuator is thus not required in such an embodiment. However, such force sensors 
can be expensive and bulky, adding to the cost and size of the interface mechanism. 

Figure 12b is a schematic diagram of an alternate transducer system 200' similar to 
transducer system 200 shown in Figure 12a. In this embodiment, sensor 210 is positioned 
between coupling 206 and object 44 on coupling shaft 208. Shaft 208 extends through sensor 
10 210 and can be rigidly coupled to object 44 at the end of the shaft. Transducer system 200* 
functions substantially the same as transducer system 200 shown in Figure 12a. 

Figure 13 is a schematic view of a preferred embodiment of transducer system 200 for a 
mechanism providing one degree of freedom that uses rotary backlash to provide play between 
actuator 202 and coupling 206. Keyed actuator shaft 214 is rigidly coupled to actuator 202 and 

15 mates with keyed coupling 216. The cross-sectional diameter of keyed actuator shaft 214 is 
preferably smaller than bore 218 of coupling 216, to provide the desired backlash, as described 
in greatef detail with reference to Figure 14a. Coupling shaft 208, sensor 210, and object 44 are 
substantially similar to these components as described with reference to Figure 12a. In alternate 
embodiments, backlash can be provided between actuator 202 and coupling 206 using different 

20 components, such as gears, pulleys, etc. 

Figure 14a is a side sectional view of keyed actuator shaft 214 and coupling 216 taken 
along line 14a- 14a of Figure 13. Keyed shaft 214 extends into keyed bore 218 of coupling 216. 
In Figure 14a. gap 220 is provided around the entire perimeter of shaft 214. In alternate 
25 embodiments, gap 220 can be provided only between the sides of the keyed portion 222 of shaft 
214, as described with reference to Figure 15. 

Figure 14b is a side sectional view of keyed actuator shaft 214 and coupling 216 taken 
along line 14b-14b of Figure 14a. Keyed shaft 214 is shown partially extending into coupling 
30 216. As shown in Figure 14a. a small gap 220 is preferably provided between coupling 216 and 
shaft 214. When shaft 214 is rotated, coupling 216 is also rotated after the keyed portion of 
shaft 214 engages the keyed portion of bore 218, as described with reference to Figure 15. 
Coupling shaft 208 rotates as coupling 216 rotates, since it is rigidly attached. 

35 Figure 15 is a detailed view of Figure 14a showing the keyed portions of shaft 214 and 

bore 218. Extended keyed portion 222 of shaft 218 protrudes into receiving keyed portion 224 
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of bore 218. In alternate embodiments, an extended keyed portion of coupling 216 can protrude 
into a receiving keyed portion of shaft 214. Gap 220 has a width d which determines how much 
desired backlash (play) is introduced between actuator 202 and object 44. (Additional 
unintentional backlash or other inherent play can exist between the components of the system due 
5 to compliance of the shafts, etc.) In the described embodiment, in which sensor 210 has a 
sensing resolution of about 1/10 degree, d is preferably about 1/1000 inch. Note that the 
distance d can widely vary in alternate embodiments. The chosen distance d is preferably made 
small enough to prevent the user from feeling the backlash that exists in the system when 
handling object 44 and yet is large enough for the sensor to detect the play (i.e., greater than the 

10 sensing resolution of sensor 2 10) to allow the sensor to inform the computer the direction that the 
user is moving object 44. Thus, the distance d is highly dependent on the sensing resolution of 
sensor 210. For example, if a sensing resolution of 1/100 degree is available, the distance d can 
be much smaller. The amount of backlash that a user can typically feel can depend on the size 
and shape of object 44; however, the backlash described above is not detectable by a user for the 

15 majority of possible objects. In other embodiments, it may be desirable to allow the user to feel 
the backlash or other play in the system, and thus a greater distance d can be implemented. 

<» 

In the preferred embodiment, distance d allows rotational movement of coupling 216 at 
least equal to the sensing resolution of sensor 210 in either direction, thus allowing a total 
20 backlash of distance of 2d between surfaces 228 and 232 of coupling 2 1 6. Alternatively, a total 
backlash of distance d between surfaces 228 and 232 can be implemented (half of the shown 
distance). In such an embodiment, however, sensor 210 would only be able to detect movement 
from one limit of the backlash to the other limit, and, for example, movement of coupling 216 
from a center position (as shown in Figure 15) would not be detected. 

25 

In the described embodiment, digital encoder sensors 210 are used, in which rotational 
movement is detected using a number of divisions on a wheel that are rotated past fixed sensors, 
as is well known to those skilled in the art. Each division causes a "pulse," , and the pulses are 
counted to determine the amount (magnitude) of movement. Distance d can be made as large or 

30 larger than the sensing resolution of the encoder so that the magnitude and direction of the 
movement within gap 220 can be detected. Alternatively, the resolution of the sensor can be 
made great enough (i.e., the distance between divisions should be small enough, in a digital 
encoder) to detect movement within gap 220. For example, two or more pulses should be able to 
be detected within distance d to determine the direction of movement of object 44 and coupling 

35 216 using a digital encoder or the like. 

When coupling 216 is initially rotated from the position shown in Figure 15 in a direction 
indicated by arrow 226 (counterclockwise in Figure 14a) as the user moves object 44, the 
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coupling freely rotates. Coupling 216 can no longer be rotated when the inner surface 228 of 
keyed portion 224 engages surface 230 of keyed portion 222. Thereafter, external force (such as 
from the user) in the same direction will cause either both coupling 216 and shaft 214 to rotate in 
the same direction, or the external force will be prevented if actuator 202 is locking shaft 214 in 
5 place with high resistive force to prevent any rotational movement of shaft 214. 

If the user suddenly moves object 44 in the opposite rotational direction after surface 228 
has engaged surface 230, coupling 216 can again be rotated freely within gap 220 until surface 
232 of bore 218 engages surface 234 of shaft 214, at which point both shaft and coupling are 

10 rotated (or no rotation is allowed, as described above). It is the magnitude and direction of the 
movement between the engagement of the surfaces of keyed portions 222 and 224 which can be 
detected by sensor 210, since sensor 210 is rigidly coupled to coupling 216. Since sensor 210 
can relay to the controlling computer the direction which coupling 216 (and thus object 44) is 
moving, the computer can deactivate or activate actuator 202 accordingly. Even if object 44 is 

15 held in place by actuator 202, as when moving into a virtual "wall", the computer can detect the 
backlash movement of object 44 if the user changes the direction of the object and can release the 
brakes accordingly. It should be noted that computer 16 should preferably deactivate (release) 
the passive actuator before surface 232 engages surface 234 so that the user, will not feel any 
resistance to movement in the opposite direction. 

20 Figure 16 is a schematic diagram of an alternate embodiment of transducer system 200 in 

which the desired play between actuator 202 and object 44 is provided by a flexible (i.e. 
compliant) coupling instead of the keyed shaft system with backlash shown in Figure 13. A 
flexible coupling can take many possible forms, as is well known to those skilled in the art. The 
flexible coupling allows coupling shaft 208 to rotate independently of actuator shaft 204 for a 

25 small distance, then forces actuator shaft 204 to rotate in the same direction as coupling shaft 
208, as described with reference to Figures 13-15. In Figure 16, actuator 202, coupling shaft 
208. sensor 210 and object 44 are similar the equivalent components as discussed above with 
reference to Figure 12a. A flexible coupling 236 has two ends 219 and lengthwise portion* 221 
that provide torsion flex between the ends 219. Flexible coupling 236 thus allows an amount of 

30 torsion flex (play) about axis H between coupling shaft 208 and actuator shaft 215. When 
actuator shaft 215 is locked in place by actuator 202, coupling shaft 208 is rotated, and coupling 
236 has been flexed to its limit in one rotational direction, shaft 208 will be prevented from 
rotating in the same direction and the user will be prevented from moving object 44 further in thai 
direction. If object 44 and coupling shaft 208 were caused to suddenly rotate in the opposite 

35 direction, coupling 236 would flex freely in that direction and this movement would be detected 
by sensor 210, allowing the computer to change resistive force applied by actuator 202 
accordingly. When coupling 236 reached maximum flexibility in the other direction, the 
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mechanism would perform similarly and the user would feel forces (if any) from actuator 202. 
Compliance or flex can also be provided with spring members and the like. 

Similar to the backlash system described in Figures 13-15, the amount of play provided 
by flexible coupling 236 between actuator 202 and object 44 is equal to or greater than the 

5 sensing resolution of sensor 210. A typical flexible coupling has an inherent amount of stiffness 
so that a force must be applied to overcome the stiffness. Preferably, flexible coupling 236 has a 
low stiffness and flexes with a small amount of force with respect to the maximum drag output 
by the passive actuator 202. Flexible coupling 236 also preferably has a small amount of flex to 
provide a small amount of desired play; as above, the desired play when using flexible coupling 

10 236 should be the minimum amount of play that the sensor 210 can reliably detect. 

Figure 17 is a schematic diagram of an embodiment of a mechanical apparatus 240 using 
transducer system 200. Similar to apparatus 25 as described with reference to Figure 2, 
apparatus 200 includes a gimbal mechanism 38 and a linear axis member 40. A user object 44 is 
preferably coupled to linear axis member 40. Gimbal mechanism 38 provides two revolute 

15 degrees of freedom as shown by arrows 242 and 244. Linear axis member 40 provides a third 
linear degree of freedom as shown by arrows 246. These components function as described 
with reference to Figure 2. Coupled to each extension member 4§a and 48b is a transducer 
system 238 (equivalent to transducer system 200) and 239 (equivalent to transducer system 
200 1 ), respectively. It should be noted that the two different embodiments of transducer system 

20 200 and 200* are shown on one mechanical apparatus 240 for illustrative purposes. Typically, 
only one embodiment of system 200 or 200' is used for both ground members 48a and 48b. 

Transducer system 238 is similar to the system shown in Figure 12a wherein object 44 is 
positioned between coupling 206 and sensor 210. Transducer system 238 includes actuator 
202a, which is grounded and coupled to coupling 206a (ground 56 is schematically shown 

25 coupled to ground member 46, similar to Figure 2). Coupling 206a is coupled to extension 
member 48a which ultimately connects to object 44 and provides a revolute degree of freedom 
about axis A. Sensor 210a is rigidly coupled to extension member 48a at the first bend 237 in 
the extension member. Sensor 210a is also grounded by either coupling it to ground member 46 
or separately to ground 56. Sensor 210a thus detects all rotational movement of extension 

30 member 48a and object 44 about axis A. However, coupling 206a provides a desired amount of 
play between actuator 202a and extension member 48a as described above. Alternatively, sensor 
210a can be rigidly coupled to extension member 48a at other positions or bends in member 48a, 
or even on central member 50b, as long as the rotation of object 44 about axis A is detected. 

Transducer system 239 is similar to the transducer system shown in Figure 12b in which 
35 sensor 210 is positioned between coupling 206 and object 44. Actuator 202b is grounded and is 
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non-rigidly coupled (i.e., coupled with the desired play as described above) to coupling 206b. 
Coupling 206b is rigidly coupled, in turn, to sensor 2 10b f which separately grounded and rigidly 
coupled to ground member 46 (leaving coupling 206b ungrounded). Extension member 48b is 
also rigidly coupled to coupling 206b by a shaft extending through sensor 210b (not shown). 
5 Sensor 210b thus detects all rotational movement of extension member 48b and object 44 about 
axis B. Coupling 206b provides a desired amount of play between actuator 202b and extension 
member 48b for reasons described above. 

Rotational resistance or impedance can thus be applied to either or both of extension 
members 48a and 48b and object 44 using actuators 202a and 202b. Couplings 206a and 206b 
10 allow computer 16 to sense the movement of object 44 about either axis A or B when actuators 
arc locking the movement of object 44. A similar transducer system to system 238 or 239 can 
also be provided for linear axis member 40 to sense movement in and provide force feedback to 
the third degree of freedom along axis C. Such a system can be implemented similarly to the 
transducers shown in Figure 6 and as described below, 

15 Figure 18 is a perspective view of a preferred embodiment of mechanical apparatus 240 

shown in Figure 17. Apparatus 240 is similar to the embodiment of apparatus 25 144 shown in 
Figure 8 above, in which object 44 is implemented as a joystick 1 12 movable in two degrees of 
freedom about axes A and B. For illustrative purposes, apparatus 240 is shown with two 
embodiments of transducer system 200 and 200'. System 239 is shown similarly as in Figure 

20 17 and includes actuator 202b, coupling 206b, and sensor 210b, with the appropriate shafts 
connecting these components not shown. Actuator 202b is grounded by, for example, a support 
member 241. The coupling shaft 208 extending from sensor 210b is preferably coupled to 
capstan pulley 76 of capstan drive mechanism 58. When object 44 is moved about axis A. 
extension member 48b is also moved, which causes capstan member 59 (which is rigidly 

25 attached to member 48b) to rotate. This movement causes pulley 76 to rotate and thus transmits 
the motion to the transducer system 239. As described above with reference to Figure 5, the 
capstan mechanism allows movement of object 44 without substantial backlash. This allows the 
introduced, controlled backlash of coupling 206 to be the only backlash in the system. In 
addition, as described previously, the capstan drive mechanism provides a mechanical advantage 

30 for the movement of object 44. Sensor 210b can thus detect rotation at a higher resolution and 
actuator 202b can provide greater forces to object 44. This can be useful when, for example, a 
user can overpower the resistive forces output by actuator 202b; capstan mechanism 58 allows 
greater forces to be output from an actuator that are more difficult for the user to overcome. A 
different type of gearing system can also be used to provide such mechanical advantage, such as 

35 a pulley system. Transducer system 239 or 238 can also be directly connected to ground 
member 46 and extension member 48a or 48b, as shown in Figure 17. For example, transducer 
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system 239 can be directly coupled to vertical support 62 and capstan member 59 on axis A. 
However, in such a configuration, the described benefits of the capstan drive mechanism would 
not be gained. 

Transducer system 238 is shown coupled to the other extension member 48a similarly as 
5 in Figure 17. In this configuration, actuator 202a and coupling 206a are positioned on one side 
of vertical support member 62. Coupling shaft 208 preferably extends through vertical support 
member 62 and pulley 76 and is coupled to sensor 2 10a, which is grounded. Transducer system 
238 gains the advantages of the capstan drive mechanism as described above. Alternatively, 
sensor 210b can be coupled to capstan member and vertical support 62 at axis B; however, the 
10 sensor would gain no mechanical advantage from the capstan drive mechanism 58 at this 
location. Actuator 202a and sensor 210b are preferably grounded by, for example, support 
members 243. 

Transducer systems 238 and 239 can also be used with other apparatuses as shown in the 
embodiments of Figures 3 and 7. For example, a third linear degree of freedom and a fourth 
15 rotational degree of freedom can be added as shown in Figure 3. Transducer systems 238 or 239 
can be used to sense movement in and provide force feedback to those third and fourth degrees 
of freedom. Similarly, transducer system 238 or 239 can be applied to the fifth and sixth degrees 
of freedom as shown and described with reference to Figure 7. 

Figure 19 is a perspective view of alternate interface apparatus 250 suitable for use with 
20 transducer system 200. Mechanism 250 includes a slotted yoke configuration for use with 
joystick controllers that is well-known to those skilled in the art. Apparatus 250 includes slotted 
yoke 252a, slotted yoke 252b, sensors 254a and 254b, bearings 255a, and 255b, actuators 256a 
and 256b, couplings 258a and 258b, and joystick 44. Slotted yoke 252a is rigidly coupled to 
shaft 259a that extends through and is rigidly coupled to sensor 254a at one end of the yoke. 
25 Slotted yoke 252a is similarly coupled to shaft 259c and bearing 255a at the other end of the 
yoke. Slotted yoke 252a is rotatable about axis L and this movement is detected by sensor 254a. 
Coupling 254a is rigidly coupled to shaft 259a and is coupled to actuator 256 such that a desired 
amount of play is allowed between actuator 265 and shaft 259a. This arrangement permits the 
play between object 44 and the actuator as described in the above embodiments. Actuator 256a is 
30 preferably a passive actuator such as magnetic particle brakes. In alternate embodiments, 
actuator 256a and coupling 258a can be instead coupled to shaft 259c after bearing 255a. In yet 
other embodiments, bearing 255a and be implemented as another sensor like sensor 254a. 

Similarly, slotted yoke 252b is rigidly coupled to shaft 259b and sensor 254b at one end 
and shaft 259d and bearing 255b at the other end. Yoke 252b can rotated about axis M and this 
35 movement can be detected by sensor 254b. A coupling 258b is rigidly coupled to shaft 259b and 



WO 96/22591 



PCT/US96/00701 



38 

an actuator 256b is coupled to coupling 258b such that a desired amount of play is allowed 
between shaft 259b and actuator 256b, similar to actuator 256a described above. 

Object 44 is a joystick 112 that is pivotally attached to ground surface 260 at one end 262 
so that the other end 264 typically can move in four 90-degree directions above surface 260 (and 

5 additional directions in other embodiments). Joystick 112 extends through slots 266 and 268 in 
yokes 252a and 252b, respectively. Thus, as joystick 1 12 is moved in any direction, yokes 252a 
and 252b follow the joystick and rotate about axes L and M. Sensors 254a-d detect this rotation 
and can thus track the motion of joystick 1 12. The addition of actuators 256a and 256b allows 
the user to experience force feedback when handling joystick 44. The couplings 258a and 258b 

10 provide an amount of play, as described above, to allow a controlling system to detect a change 
in direction of joystick 112, even if joystick 112 is held in place by actuators 256a and 256b. 
Note that the slotted yoke configuration typically introduces some inherent play (such as 
compliance or backlash) to the mechanical system. Couplings 259a and 259b can be added to 
provide an additional amount of play, if desired. Similarly, other interface apparatuses thai 

15 typically provide an amount of inherent play can be used such that the inherent play is measured 
by sensor 210 and no coupling 206 is required. Also, other types of objects 44 can be used in 
place of joystick 1 12, or additional objects can be coupled to joystick 1 12. 

In alternate embodiments, actuators and couplings can be coupled to shafts 259c and 
259d to provide additional force to joystick 1 12. Actuator 256a and an actuator coupled to shaft. 
20 259c can be controlled simultaneously by a computer or other electrical system to apply or release 
force from bail 252a. Similarly, actuator 256b and an actuator coupled to shaft 259d can be 
controlled simultaneously. 

Figure 20 is a block diagram 270 of an electronic interface suitable for use with the 
transducer system 200. The electronic components in diagram 270 are preferably used with 
25 passive actuators and optical encoder sensors. The interface of diagram 270, however, can alsci 
be used with other embodiments of interface apparatus 25 as described above. 

Host computer 16 can be computer system 16 as described above with reference tc 
Figures 1 and 9 and is preferably implements a simulation or similar virtual environment which 
user is experiencing and moving object 44 in response to, as is well known to those skilled in the 

30 art. Host computer 16 includes interface electronics 272. In the described embodiment, interfaoi 
electronics include a serial port, such as an RS-232 interface, which is a standard interface 
included on most commercially available computers. This interface is different than the interface 
card and electronics shown with respect to Figure 9 above, which allows faster control signa 
transmission and is thus more suitable for controlling active actuators than the presentl; 

35 described interface electronics. 
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Microprocessor 274 can be used to control input and output signals that are provided to 
and from interface 272. For example, microprocessor can be provided with instructions to wait 
for commands or requests from computer host 16, decode the command or request, and handle 
input and output signals according to the command or request. If computer 16 sends a command 
5 to control actuators, microprocessor 274 can output signals to the actuator representing the force 
to be applied by the actuator, and can send an acknowledgment to computer 16 that such output 
was sent. If computer 16 sends a request for sensory input, microprocessor 274 can read 
position data from the sensors and send this data to the computer 16. Suitable microprocessors 
for use as microprocessor 274 include the MC68HC711E9 by Motorola and the PIC16C74 by 
10 Microchip. 

Digital-to-analog converter (DAQ 276 is electrically coupled to microprocessor 274 and 
receives digital signals representing a force value from the microprocessor. DAC 276 converts 
the digital signal to analog signal as is well known to those skilled in the art. A suitable DAC is 
the MAX530ACNG manufactured by Maxim. Power amplifier 278 receives the analog signal 

15 from DAC 276 and converts the signal into an appropriate brake control signal for actuator 202. 
For example, an LM324 and TIP31 can be used as power amplifier 278. Actuator 202, which is, 
preferably a magnetic particle brake by Force Limited, Inc., receives the brake signal and 
provides appropriate resistive forces to impede the motion of object 44 caused by the user. 
Preferably, a separate DAC and power amplifier is used for each actuator 202 implemented in the 

20 interface apparatus so the computer 16 can control each actuator separately for each provided 
degree of motion. 

Sensor 210 is preferably a digital optical encoder which operates as described above; for 
example, a suitable encoder is the "Softpot" from U.S. Digital of Vacouver, Washington. The 
sensor detects the position of object 44 and provides a digital position signal to decoding 

25 electronics 280, which convert the sensor signal into an input signal suitable to be interpreted by 
computer 16. For example, quadrature decoder LS7 166 is suitable to decode quadrature signals 
from sensor 210. The position value signals are interpreted by computer 16 which updates the 
virtual reality environment and controls actuator 202 as appropriate. Other interface mechanisms 
other than decoding electronics 280 can also be used to provide an appropriate signal to 

30 microprocessor 274. In alternate embodiments, an analog sensor 210 can be used to provide an 
analog signal to an analog-to-digital converter (ADC), which can provide a digital position signal 
to computer 16. The resolution of the detected motion of object 44 would then be limited by the 
resolution of the ADC. However, noise can sometimes mask small movements of object 44 
from an analog sensor 210, which can potentially mask the play that is important to the present 

35 embodiment of the invention. 
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Figure 21 is a flow diagram illustrating the control process 300 of actuator 202 during an 
example of simulated motion of object 44 along one degree of freedom through a fluid or similar 
material. Process 300 can be implemented by computer 16 or by microprocessor 274 in 
conjuction with computer 16. The process starts at 310, and, in step 312, a damping constant is 
5 initialized. This constant indicates the degree of resistance that object 44 experiences when 
moving through a simulated material, where a greater number indicates greater resistance. For 
example, water would have a lower damping constant than oil or syrup. 

In step 314, the current position of object 44 along the examined degree of freedom is 
stored in a variable X0. In step 316, the current position of object 44 along the examined degree 

10 of freedom is stored in a variable XI. When process 300 is initially implemented, X0 and XI 
are set to the same value. In step 3 18, a variable AX is set to the difference between X 1 and X0 
(which is zero the first time implementing the process). From the sign (negative or positive) of 
AX, the direction of the movement of object 44 can also be determined. In next step 320, a 
variable FORCE is set equal to the damping constant multiplied by AX. A signal representative 

15 of the value of FORCE is then sent to the brake (or other passive actuator) in step 322 to set the 
brake impedance at the desired level. In step 324, variable X0 is set equal to XI, and the process 
then returns to step 3 16 to read and store another position of object 44 in variable X 1 . Process 
300 thus measures the manual velocity of object 44 as controlled by the user and produces a 
brake impedance (FORCE) proportional to the user's motion to simulate movement through a 

20 fluid. Movement in other mediums, such as on a bumpy surface, on an inclined plane, etc., can 
be simulated in a similar fashion using different methods of calculating FORCE. 

Figure 22 is a flow diagram 328 illustrating a preferred method of modelling a "wall" or 
other hard surface or obstruction in a virtual environment when using a mechanical interface such 
as interface 240 or interface 250 with transducer system 200. It is assumed for this method that 
25 an object 44 is being grasped and moved by a user in a virtual environment. A computer system 
16 is preferably detecting the position of the object and providing force feedback to the object 
when appropriate. 

The method starts at 330, and, in a step 332, the position of an object is sensed by the 
computer 16 and/or microprocessor 274. Sensors 210 provide the rotary and/or linear position 

30 of object 44 in the number of degrees of freedom being sensed. The computer 16 updates a 
virtual reality environment in response to the user's movements of object 44. For example, if the 
user moves a steering wheel object 44, the computer 16 can move the point of view of the user as 
if looking out a vehicle and turning the vehicle. It should be noted that the computei 
16/microprocessor 274 can be providing force feedback to the user that is not related to the 

35 virtual wall in this step as well. For example, the computer can cause a joystick to require greatei 
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force to be moved when simulating a vehicle moving in mud, over a bumpy surface, etc., as 
described above with reference to Figure 21. 

In step 334, it is determined if object 44 (or a virtual, computer-generated object 
controlled by object 44) has been moved into a virtual wall or a similar obstruction that can 
prevent object 44 from moving in one or more directions. If the object has not been moved into 
such an obstruction, step 272 is repeated and any other appropriate force feedback according to 
the object's movement can be applied. If the object has been moved into such an obstruction, 
then step 336 is implemented, in which the passive actuator such as a brake provides maximum 
impedance to the motion of object 44 along the obstructed degree(s) of freedom. This feels to the 
user as if the object 44 has hit an obstruction and can no longer be moved in the direction of the 
"wall" or obstacle. 

In next step 338, the computer 16 checks for any movement in direction opposite to the 
wall. If no movement in this direction is sensed by sensors 210, then continued maximum 
resistive force is applied to object 44 in step 336; the user is thus still forcing object 44 towards 
the wall. If the computer/microprocessor detects movement away from the wall in step 338, due 
to the play caused by coupling 206, then step 340 is implemented, in which the 
computer/microprocessor releases the brakes before the limit to the play is reached in the new 
direction (i.e., within the allowed compliance or backlash). The user can thus freely move object 
44 away from the wall without feeling like it is stuck to the wall. The process then returns to 
step 332, in which the computer/microprocessor senses the position of object 44. 

While this invention has been described in terms of several preferred embodiments, it is 
contemplated that alterations, modifications and permutations thereof will become apparent to 
those skilled in the art upon a reading of the specification and study of the drawings. For 
example, the linked members of apparatus 25 can take a number of actual physical sizes and 
forms while maintaining the disclosed linkage structure. In addition, other gimbal mechanisms 
can also be provided with a linear axis member 40 to provide three degrees of freedom. 
Likewise, other types of gimbal mechanisms or different mechanisms providing multiple degrees 
of freedom can be used with the capstan drive mechanisms disclosed herein to reduce inertia, 
friction, and backlash in a system. A variety of devices can also be used to sense the position of 
an object in the provided degrees of freedom and to drive the object along those degrees of 
freedom. In addition, the sensor and actuator used in the transducer system having desired play 
can take a variety of forms. Similarly, other types of couplings can be used to provide the 
desired play between the object and actuator. Furthermore, certain terminology has been used 
for the purposes of descriptive clarity, and not to limit the present invention. It is therefore 
intended that the following appended claims include all such alterations, modifications and 
permutations as fall within the true spirit and scope of the present invention. 
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What is claimed is: 

Claims 

I. An apparatus for interfacing motion of a user-manipulable object with an electrical 
system comprising: 

5 a gimbal mechanism including a plurality of members and providing two rcvolute degrees 

of freedom to a user-manipulable object about two axes of rotation; 

a linear axis member translatably coupled to said gimbal mechanism and independently 
translatable with respect to said gimbal mechanism along a linear third axis in a third degree of 
freedom, wherein said object is coupled to said linear axis member and is independently 
10 translatable along said third axis with respect to said gimbal mechanism, said third axis extending 
approximately through an intersection of said two axes of rotation; and 

a first degree of freedom transducer coupled between two of said members of said gimbal 
mechanism; 

whereby said transducer provides an electromechanical interface between said object and 
1 5 said electrical system. 



2. An apparatus as recited in claim 1 wherein said gimbal mechanism includes a closed 
loop five member linkage, wherein each of said five members is rotatably coupled to at least twc 
other members of said linkage. 

20 

3. (amended) An apparatus as recited in claim 2 wherein said five member linkage 
includes: 

a ground member coupled to a ground surface; 

first and second extension members, each extension member being coupled to sak 
25 ground member, 

first and second central members, said first central member having an end coupled to sail 
first extension member and said second central member having an end coupled to said secont 
extension member, wherein said central members are coupled to each other at ends not coupled t< 
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said extension members and wherein at least one of said central members is transiatably coupled 
to said linear axis member. 

4. An apparatus as recited in claim 3 wherein said two axes of rotation are fixed in one 
position with respect to said ground member of said gimbal apparatus, and wherein said first 

5 central member is rotatable about a first a floating axis and said second central member is 
rotatable about a second floating axis, said first and second floating axes being moveable with 
respect to said ground member of said gimbal apparatus. 

5. An apparatus as recited in claim 3 wherein said first extension member, said first 
central member, and said second central member are rotatable about said first axis, and said 

10 second extension member, said second central member, and said first central member are 
rotatable about said second axis. 

6. An apparatus as recited in claim 1 wherein said transducer includes an actuator for 
imparting a force along said first degree of freedom in response to electrical signals produced by 
said electrical system, and wherein said transducer includes a sensor for sensing positions of said 

15 object along said first degree of freedom and which produces electrical signals corresponding to n 
such positions for said electrical system. 

7. An apparatus as recited in claim 1 further comprising: 

a second degree of freedom transducer coupled between two of said members of said 
gimbal mechanism: and 

20 a third degree of freedom transducer coupled between said linear axis member and said 

gimbal mechanism. 

8. An apparatus as recited in claim 7 wherein said linear axis member provides a fourth 
degree of freedom about a fourth axis of rotation, said fourth axis of rotation being parallel to 
said linear axis member, and further comprising a fourth degree of freedom transducer coupled 

25 between said linear axis member and said object. 

9. An apparatus as recited in claim 8 further comprising a floating gimbal mechanism 
coupling said linear axis member to said object to provide rotational movement for said object 
along a fifth degree of freedom. 

10. An apparatus as recited in claim 9 further comprising a fifth degree of freedom 
30 transducer coupled between said linear axis member and said floating gimbal mechanism. 
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ll. An apparatus as recited in claim 10 wherein said object is rotatable about a 
longitudinal sixth axis of said object to provide a sixth degree of freedom for said object, and 
further comprising a sixth degree of freedom transducer coupled between said object and said 
gimbal mechanism. 

5 12. An apparatus as recited in claim 7 wherein said linear axis member includes a sleeve, 

and wherein said object includes a shaft rotatable within said sleeve. 

13. An apparatus as recited in claim 10 wherein said object includes one of a stylus, at 
least a portion of a surgical instrument operative to be grasped by a user, and a joystick. 

14. An apparatus as recited in claim 13 wherein said portion of a surgical instrument 
10 includes one of a portion of a hypodermic needle and a portion of a laproscopic surgical tool. 



15. An apparatus for interfacing the motion of a user-manipulable object with an electrical 
system comprising: 

a gimbal mechanism providing two revolute degrees of freedom to an object engaged 
15 with said gimbal mechanism about two axes of rotation, said object being coupled to said gimbal 
mechanism at about the intersection of said two axes of rotation; 

a transducer coupled to said gimbal mechanism to generate a force along said first degree 
of freedom; and 

a capstan drive mechanism coupled between said actuator and said gimbal mechanism, 
20 said capstan drive mechanism including a capstan drum coupled to a pully by a flexible member, 
said flexible member transmitting said force generated by said transducer to said gimbal 
mechanism and transmitting forces applied by a user from said gimbal mechanism to said 
transducer: 

whereby said transducer provides an electromechanical interface between said object and 
25 said electrical system. 

16. An apparatus as recited in claim 15 further comprising a linear axis member coupled 
to said gimbal mechanism at said intersection of said two axes of rotation capable of being 
translated independendy with respect to said gimbal mechanism along a third axis in a third 
degree of freedom, wherein said object is coupled to said linear axis member and is translatable 

30 along said third axis. 
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17. An apparatus as recited in claim 15 wherein said gimbal mechanism includes a closed 
loop five member linkage, wherein each of said five members is rotatably coupled to at least two 
other members of said linkage. 

18. An apparatus as recited in claim 17 wherein said two axes of rotation are decoupled 
5 from each other such that said object is movable about either of said axes of rotation without 

moving said other axis of rotation. 

19. An apparatus as recited in claim 15 wherein said capstan drum is rotatably coupled to 
a ground member of said gimbal mechanism and is rigidly coupled to a different member of said 
gimbal mechanism, and wherein said flexible member is a cable, said pully is rigidly coupled to 

10 said transducer, and said transducer is operative to rotate said pulley and thereby transmit force to 
said gimbal mechanism with no substantial backlash. 

20. An apparatus as recited in claim 15 further comprising a second transducer and a 
second capstan drive mechanism coupled between said second transducer and said gimbal 
mechanism, said second capstan drive mechanism transmitting a force generated by said second 

15 actuator to said gimbal mechanism in a second degree of freedom and transmitting a force applied 
to said gimbal mechanism by a user to said transducer. , 

21. An apparatus as recited in claim 16 further comprising a third axis capstan drive 
mechanism for transmitting forces between said third degree of freedom transducer and said 
linear axis member. 

20 22. An apparatus as recited in claim 20 further comprising a plurality of sensors coupled 

to said apparatus to sense positions of said object along said two degrees of freedom and which 
produce electrical signals corresponding to such positions for said electrical system. 

23. An apparatus as recited in claim 22 wherein said object includes one of the groups 
consisting of at least a portion of a surgical tool, a stylus, and a joystick. 

25 24. An apparatus as recited in claim 17 wherein said apparatus includes sensors for 

sensing positions of said object along fourth, fifth and sixth degrees of freedom, respectively. 

25. A method for interfacing motion of an object with an electrical system, the method 
comprising the steps of: 

30 defining an origin in 3-dimensional space; 
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providing a gimbal mechanism movable relative to said origin in said 3-dimensional space 
such that an object engaged with said gimbal mechanism has first and second degrees of 
freedom; 

providing a translatable member that couples said object to said gimbal mechanism to 
5 independently translate said object with respect to said gimbal mechanism in a third degree of 
freedom; 

transducing an electrical signal for each of said three degrees of freedom using first, 
second, and third transducers, respectively, wherein two of said transducers are decoupled from 
each other, and 

10 providing said electrical signals to and from said electrical system. 



26. A method as recited in claim 25 wherein said gimbal mechanism is a closed loop five 
member linkage, wherein each of said five members is rotatably coupled to at least two other 
members of said linkage. 

IS 27. A method as recited in claim 26 wherein said step of transducing an electrical signal 

for each of said three degrees of freedom includes converting output signals from said digital 
processing system into movement of said object. 

28. A method as recited in claim 27 further comprising a step of transducing an electrical 
signal for a fourth degree of freedom of said object using a fourth transducer, said fourth degree 

20 of freedom being provided by rotating said object and said translatable member about a 
transverse axis of said translatable member. 

29. A method as recited in claim 28 further comprising a step of transducing an electrical 
signal for fifth and sixth degrees of freedom of said object usin? fifth and sixth transducers, 
respectively. 

25 30. A method as recited in claim 28 further comprising transmitting a force from an 

actuator included in one of said transducers to said gimbal mechanism using a capstan drive 
mechanism, said capstan drive mechanism including a cable and pully for transmitting said force. 

3 1. An apparatus as recited in claim 25 wherein said gimbal mechanism is grounded. 
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32. An apparatus for interfacing motion of a user-manipulable object with an electrical 
system comprising: 

a gimbaJ mechanism including a closed loop linkage of five members, wherein each of 
said five members is rotatably coupled to at least two other members of said linkage; 

S a user-manipulable object coupled to said gimbal mechanism such that said gimbal 

mechanism provides two revolute degrees of freedom to said object about two axes of rotation, 
said object being coupled to said gimbal mechanism at about an intersection of said two axes of 
rotation and being independently rbtatable with respect to said gimbal mechanism about a third 
axis in a third degree of freedom, said third axis extending approximately through said 

10 intersection of said two axes of rotation; and 

first and second degree of freedom transducers, wherein each of said transducers is 
coupled between a ground member of said gimbal mechanism and a different members of said 
gimbal mechanism; 

whereby said transducer provides an electromechanical interface between said object and 
15 said electrical system. 

33. An apparatus as recited in claim 32 wherein said transducer includes an actuator for 
transmitting a force to said object about one of said revolute degrees of freedom. 

34. An apparatus as recited in claim 32 wherein said object is at least a portion of a 
joystick handle. 

20 35. An apparatus as recited in claim 32 wherein said five members each includes two 

ends, and wherein each of said five members is rotatably coupled to a different other one of said 
five members at each of said ends. 

36. An apparatus as recited in daim 32 wherein said linkage of five members includes: 

a ground member, 

25 first and second extension members, each extension member being coupled to said 

ground member, 

first and second central members, said first central member having an end rotatably 
coupled to said first extension member and said second central member having an end rotatably 
coupled to said second extension member, wherein said central members are coupled to each 
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other at ends not coupled to said extension members and wherein at least one of said central 
members is coupled to said object. 

37. An apparatus as recited in claim 36 wherein said two axes of rotation are fixed with 
respect to said ground member, said first and second extension members being rotatable about 
said fixed axes of rotation, and wherein said central members are rotatable about first and second 
floating axes, said floating axes being movable with respect to said ground member. 

38. An apparatus as recited in claim 32 further comprising a linear axis member rotatably 
coupled to said gimbal mechanism, wherein said object is coupled to said linear axis member 
such that said object and said linear axis member are rotatable about a third axis with respect to 
said gimbal mechanism. 

39. An interface apparatus for interfacing motion of a user manipulable object with a host 
computer system displaying visual images on a screen, said interface apparatus comprising: 

a user object movable in a degree of freedom by a user and being physically contacted by 
said user; 

a gimbal mechanism coupled to said user object and providing two revolute degrees of 
freedom to said user object about two axes of rotation, said gimbal mechanism including a 
closed-loop linkage of a plurality of members; 

a processor, separate from said host computer system, for communicating with said host 
computer system via a communication interface by receiving a host command from said host 
computer system, said processor being controlled by software instructions; 

an actuator electrically coupled to said processor for applying a force along a degree of 
freedom to said user object in accordance with a processor command from said processor, said 
processor command being derived from said host command; and 

a sensor for detecting a position of said user object along said degree of freedom and 
outputting said input signals to said host computer system, said input signals including 
information representative of said position of said user object. 

40. An interface apparatus as recited in claim 39 wherein said closed loop linkage 
includes five members, and wherein each of said five members of said closed loop linkage is 
rotatably coupled to at least two other members of said linkage. 
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41. An interface apparatus as recited in claim 40 further comprising a linear axis member 
coupled to said gimbal mechanism and being independently rotatable with respect to said gimbal 
mechanism about a third axis in a third degree of freedom, said third axis extending 
approximately through said intersection of said two axes of rotation, wherein said object is 

5 coupled to said linear axis member and is independently rotatable about said third axis with 
respect to said gimbal mechanism. 

42. An interface device as recited in claim 39 wherein said sensor is electrically coupled 
to said processor, wherein said sensor outputs said input signals to said processor, and wherein 
said processor sends said input signals to said host computer system. 

10 43. An interface device as recited in claim 42 wherein said communication interface 

includes a serial interface. 

44. An interface device as recited in claim 40 wherein said object is a joystick handle. 

45. An interface device as recited in claim 40 wherein said actuator is coupled to a 
ground member, and further comprising an additional actuator coupled to a ground member of, 

15 said gimbal mechanism, said additional actuator being operative to apply a force along a degree 
of freedom to said user object in response to signals received from said processor. 



46. An apparatus for interfacing the motion of an object with an electrical system 
comprising: 

20 a sensor for detecting movement of said object along a degree of freedom, said sensor 

having a sensing resolution and being operative to develop an electrical sensor signal for an 
electrical system; and 

an actuator assembly coupled to said object, said actuator assembly including: 

an actuator for transmitting a force to said object along said degree of freedom, 
25 said actuator being responsive to an electrical actuator signal provided by said electrical 

system; and 

a play mechanism coupled between said actuator and said object for providing a 
desired amount of play between said actuator and said object along said degree of 
freedom, wherein said desired amount of play is equal to or greater than said sensing 
30 resolution. 
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such thai said actuator and said sensor provide an electromechanical interface between said 
object and said electrical system. 

47. An apparatus as recited in claim 46 wherein said actuator includes a passive acutator 
5 and wherein said force transmitted to said object includes drag transmitted to said object to resist 

a movement of said object. 

48. An apparatus as recited in claim 47 wherein an amount of play less than said sensing 
resolution exists between said sensor and said object. 

49. An apparatus as recited in claim 48 wherein said play between sensor and object is at 
10 least an order of magnitude less than said sensing resolution. 

50. An apparatus as recited in claim 49 wherein said sensor is rigidly coupled to said 
mechanism. 

51. An apparatus as recited in claim 50 wherein said passive actuator includes a magnetic 
particle brake. 

15 52. An apparatus as recited in claim 50 wherein said play includes rotary backlash. 

53. An apparatus as recited in claim 52 wherein said rotary backlash is provided by a 
coupling coupled to said object and having a keyed bore which is smaller than a keyed shaft that 
is received by said keyed bore, said keyed shaft being coupled to said actuator. 

54. An apparatus as recited in claim 50 wherein said play includes torsion flex. 

20 55. An apparatus as recited in claim 49 further comprising a gimbal mechanism coupled 

between said object and said actuator assembly . 

56. An apparatus as recited in claim 49 further comprising a slotted yoke mechanism 
coupled between said object and said actuator assembly and between said object and said sensor. 

25 57. An apparatus for interfacing the motion of an object with an electrical system 

comprising: 

a gimbal mechanism providing a first revolute degree of freedom to an object engaged 
with said gimbal mechanism about a first axis of rotation; 
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a sensor electrically coupled to said electrical system and rigidly mechanically coupled to 
said gimbal mechanism for sensing positions of said object along said first degree of freedom, 
said sensor having a sensing resolution; and 

a braking mechanism electrically coupled to said electrical system and rigidly 
5 mechanically coupled to said gimbal mechanism to create a drag along said first degree of 
freedom, said braking mechanism providing a desired amount of play between said actuator and 
said object along said degree of freedom, said desired amount of play being equal to or greater 
than said sensing resolution; 

whereby said braking mechanism and said sensor provide an electromechanical interface 
1 0 between said object and said electrical system. 

58. An apparatus as recited in claim 57 wherein said gimbal mechanism provides a 
second degree of freedom to said object about a second axis of rotation, and further comprising a 
sensor for sensing positions of said object along said second degree of freedom and a braking 
mechanism to create a drag along said second degree of freedom. 

15 59. An apparatus as recited in claim 58 further comprising a capstan drive mechanism 

coupled between said braking mechanism and said gimbal mechanism, said capstan drive 
mechanism transmitting said drag generated by said braking mechanism to said gimbal 
mechanism and transmitting movement of said object to said sensor. 

60. An apparatus as recited in claim 59 wherein said braking mechanism includes an 
20 acutator and a coupling, said coupling providing said desired amount of play. 

61. An apparatus as recited in claim 57 further comprising a third degree of freedom 
transducer coupled between said gimbal mechanism and said object to create a drag along said 
third degree of freedom and sense movement of said object in said third degree of freedom. 

62. An apparatus as iecited in claim 60 wherein said gimbal mechanism includes a 
25 closed loop five member linkage. 

63. An apparatus as recited in claim 58 wherein said electrical system includes a host 
computer system including a display screen for displaying visual images. 

64. An apparatus as recited in claim 62 wherein said object includes one of the group 
consisting of at least a portion of a surgical tool, a stylus, and a joystick. 

30 65. An apparatus as recited in claim 63 wherein said braking mechanism includes 

magnetic panicle brakes. 
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66. An apparatus as recited in claim 61 wherein said apparatus includes a sensor for 
sensing positions of said object along fourth, fifth, and sixth degrees of freedom, said fourth 
degree of freedom being rotation about said translational axis of said linear axis member, said 
fifth degree of freedom being provided as rotation of said object about an end of said linear axis 

5 member, and said sixth degree of freedom being provided as rotation about a longitudinal axis of 
said object. 

67. A method for interfacing motion of an object with an electrical system, the method 
comprising the steps of: 

10 defining an origin in 3-dimensional space; 

providing a gimbal mechanism movable relative to said origin in said 3-dimensional space 
such that an object engaged with said gimbal mechanism has at least one degree of freedom; 

sensing positions of said object along said degree of freedom with a sensor and 
producing electrical sensor signals therefrom, wherein an amount of play less than a sensing 
IS resolution of said sensor is allowed between said sensor and said object; and 

creating a drag from a brake along said degree of freedom in response to electrical drag 
signals, wherein a desired amount of play is allowed between said brake and said object before 
said force is generated along said degree of freedom, wherein said desired amount of play is 
greater than or equal to said sensing resolution of said sensor. 

20 

68. A method as recited in claim 67 wherein said play consists of one of the group 
consisting of rotary backlash and torsion flex. 

69. A method as recited in claim 68 further comprising the step of providing a 
translatable member to move said object in a third degree of freedom. 

25 70. A method as recited in claim 67 wherein said step of creating a drag includes 

activating a magnetic particle brake. 



71. A system for controlling an electromechanical interface apparatus manipulated by a 
user, the system comprising: 
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a digital computer system for receiving an input control signal and for providing an 
output control signal, wherein said digital computer system updates a process in response to said 
input control signal; 

a passive actutator for receiving said output control signal and providing a resistive force 
5 along a degree of freedom to an object coupled to said passive actuator, said resistive force being 
based on information in said output control signal and resisting a force applied to said object by 
said user along said degree of freedom; and 

a sensor for detecting motion of said object and outputting said input control signal 
including information representative of the position and motion of said object to said digital 
10 computer system. 



72. A system as recited in claim 71 wherein said process updated by said digital 
computer system includes simulation software, wherein said digital computer displays a view of 
a simulation to said user, and wherein said user can interact with said simulation by manipulating m 

15 said object. 

73. A system as recited in claim 71 wherein said passive actuator includes a magnetic 
particle brake. 

74. A system as recited in claim 72 further comprising a play mechanism for providing a 
desired amount of play between said actuator and said object, said desired amount of play being 

20 greater than a sensing resolution of said sensor. 

75. A system as recited in claim 74 wherein said object is grasped and moved by said 

user. 

76. A system as recited in claim 71 further comprising a serial interface for ^utputting 
said output control signal from said computer system and for receiving said input control signal 

25 to said computer system. 

77. A system as recited in claim 76 further comprising a microprocessor separate from 
said digital computer system for providing said ouput control signal from said serial interface to 
said digital to analog converter and for receiving said input control signal from said sensor. 



30 



78. A method for controlling an interface apparatus, the method comprising the steps of: 
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(a) sensing a current position of an object coupled to an interface apparatus; 

(b) determining the difference between said current position of said object and a previous 
position of said object; 

(c) determining a magnitude of a resistive force to be applied to said object, said 
5 magnitude being based at least in part on said difference of said current postion and said previous 

position; 

(d) providing a control signal to a passive actuator to transmit a resistive force having said 
determined magnitude to said object; and 

10 79. A method as recited in claim 78 wherein said step of sensing the current position of 

an object includes sensing said current position of an object when said object is locked into a 
position by said passive actuator. 

80. A method as recited in claim 79 further comprising determining a damping constant 
to be multiplied by said difference to determine said magnitude of said resistive force. 

15 
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